-- Logs begin at Wed 2022-07-20 14:33:59 UTC, end at Wed 2022-07-20 15:54:15 UTC. -- Jul 20 15:11:06.118791 dv-cinder-819149-12-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 20 15:11:06.128061 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 20 15:11:06.129905 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 20 15:11:06 2022] *** Jul 20 15:11:06.129905 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 20 15:11:06.129905 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 20 15:11:06.129905 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: nodename: dv-cinder-819149-12-iscsi Jul 20 15:11:06.129905 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: machine: x86_64 Jul 20 15:11:06.129905 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: clock source: unix Jul 20 15:11:06.129905 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: pcre jit disabled Jul 20 15:11:06.129905 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: detected number of CPU cores: 4 Jul 20 15:11:06.129905 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: current working directory: / Jul 20 15:11:06.129905 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: detected binary path: /usr/bin/uwsgi-core Jul 20 15:11:06.130498 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: your processes number limit is 47680 Jul 20 15:11:06.130498 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: your memory page size is 4096 bytes Jul 20 15:11:06.130498 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: detected max file descriptor number: 2048 Jul 20 15:11:06.130498 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: lock engine: pthread robust mutexes Jul 20 15:11:06.130498 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: thunder lock: enabled Jul 20 15:11:06.130498 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 20 15:11:06.130498 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 20 15:11:06.173781 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: Python main interpreter initialized at 0x556f5d3879c0 Jul 20 15:11:06.173781 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: python threads support enabled Jul 20 15:11:06.173781 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: your server socket listen backlog is limited to 100 connections Jul 20 15:11:06.173781 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: your mercy for graceful operations on workers is 90 seconds Jul 20 15:11:06.174155 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: mapped 403077 bytes (393 KB) for 2 cores Jul 20 15:11:06.174233 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: *** Operational MODE: preforking *** Jul 20 15:11:06.174489 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: *** uWSGI is running in multiple interpreter mode *** Jul 20 15:11:06.174489 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: spawned uWSGI master process (pid: 80496) Jul 20 15:11:06.174685 dv-cinder-819149-12-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 20 15:11:06.175587 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: spawned uWSGI worker 1 (pid: 80508, cores: 1) Jul 20 15:11:06.175587 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: spawned uWSGI worker 2 (pid: 80509, cores: 1) Jul 20 15:11:06.175745 dv-cinder-819149-12-iscsi devstack@keystone.service[80496]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 20 15:11:11.622936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 15:11:11.622936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: from cryptography.utils import int_from_bytes Jul 20 15:11:11.623711 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 15:11:11.623711 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: from cryptography.utils import int_from_bytes Jul 20 15:11:11.624002 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 15:11:11.624002 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: from cryptography.utils import int_from_bytes Jul 20 15:11:11.624455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 15:11:11.624455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: from cryptography.utils import int_from_bytes Jul 20 15:11:11.672711 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 15:11:11.672910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 15:11:11.673124 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] command line args: [] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 15:11:11.673552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 15:11:11.673743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ================================================================================ {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 15:11:11.673992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] admin_token = **** {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.674239 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] config_dir = [] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.674456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.674642 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] config_source = [] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.674825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] debug = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.675026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.676186 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.676186 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.676186 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.676186 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.676428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] list_limit = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.676428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] log_config_append = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.676621 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.676780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] log_dir = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.676959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] log_file = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.677193 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.677439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.678252 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.678540 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.678723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.678902 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.679145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.679390 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.679480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 15:11:11.679620 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 15:11:11.679698 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.679841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] command line args: [] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 15:11:11.679966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.680072 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.680159 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 15:11:11.680305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.680384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ================================================================================ {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 15:11:11.680552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.680630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] admin_token = **** {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.680797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.680888 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] config_dir = [] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.681055 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.681136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.681260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] config_source = [] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.681475 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] debug = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.681565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.78/identity {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.681703 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.681813 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] publish_errors = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.681948 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.682037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.682155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.682234 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.682374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.682473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.682624 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.682758 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.682843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.683013 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] list_limit = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.683092 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] standard_threads = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.683222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] log_config_append = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.683311 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.683434 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.683513 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.683641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] log_dir = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.683719 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.683854 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] log_file = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.684047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] use_journal = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.684047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] use_json = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.684316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] use_stderr = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.684422 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] use_syslog = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.684497 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.684574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.684812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.684894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.685047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.685131 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.685308 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.685414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.685548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.685632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.685794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.685892 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.686033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.686119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.686299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.686391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.686549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.686628 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.686799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.686899 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.687045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.687180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.687321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.687517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.687593 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.687762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.688440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.688515 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.688592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.688664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.688812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.688887 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.688887 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.689024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.78/identity {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.689024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] publish_errors = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.689167 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.689239 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.689309 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.689413 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.689497 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.689580 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.689687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.689824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.689941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.690058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.690204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.690336 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] standard_threads = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.690484 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.690570 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.690735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.690814 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.690930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.691019 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.691092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] use_journal = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.691183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.691261 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] use_json = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.691414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.691499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] use_stderr = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.691579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] use_syslog = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.691651 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.691805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 15:11:11.691883 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.692062 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.692129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.692300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.692429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.692554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.692631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.692787 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.692866 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.693033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.693110 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.693267 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.693371 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.694084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.694250 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.694358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.694437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.694568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.694648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.694806 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.694897 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.695046 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.695124 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.695278 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.695360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.695507 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.695593 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.695740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.695817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.695979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.696063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.696221 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.696305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.696451 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.696531 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.696690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.696768 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.696920 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.697035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.697158 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.697272 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.697376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.697472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.697628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.697740 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.697845 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.697941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.698080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.698221 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.698393 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.698506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.698586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.698647 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.698780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.698850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.699015 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.699145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.699228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.699300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.699478 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.699560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.699716 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.699811 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.699940 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.700023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.700167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.700245 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.700409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.700492 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.700646 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.700728 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.700876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.700956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.701110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.701194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.701334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.701419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.701565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.701653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.701796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.701874 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.702022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.702143 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.702247 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.702386 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.702494 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.702572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.702722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.702803 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.702955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.703023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.703198 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.703321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.703526 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.703603 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.703722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.703799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.703879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.703952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.704077 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.704155 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.704300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.704384 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.704542 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.704621 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.704773 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.704858 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.705009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.705094 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.705249 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.705333 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.705481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.705561 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.705715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.705792 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.705946 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.706024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.706172 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.706247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.706430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.706509 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.706664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.706759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.706896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.706984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.707125 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.707202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.707357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.707436 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.707589 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.707673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.707821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.707899 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.708050 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.708129 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.708280 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.708365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.708515 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.708610 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.708763 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.708841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.709037 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.709125 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.709262 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.709341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.709497 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.709581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.709730 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.709817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.709964 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.710043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] auth.external = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.710192 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.710303 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.710455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.710546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.710691 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.710770 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.710936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.711014 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] auth.password = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.711170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] auth.token = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.711260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.711401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.711478 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.711634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.711718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.711864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.711949 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.712095 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.712174 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.712339 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.712416 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.712579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.712665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.712805 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.712883 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.713040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.713117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] credential.caching = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.713278 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.713365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.713512 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.713591 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.713746 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.713825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.713979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.714090 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.714204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.714327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.714454 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.714534 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.714681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.714810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.714914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.714996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.715133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.715212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.715371 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.715451 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.715603 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.715695 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.715847 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.715932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.716080 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.716159 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.716330 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.716417 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.716585 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.716673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.716822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.716901 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] federation.caching = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.717048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] auth.external = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.717135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.717275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.717360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.717567 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.717647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.717797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.717887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.718023 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] auth.password = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.718101 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.718249 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] auth.token = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.718374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.718504 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.718597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.718737 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.718818 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.718978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.719058 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.719204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.719354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.719430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.719503 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.719676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.719756 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] identity.caching = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.719902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.719990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.720133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] credential.caching = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.720213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.720375 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.720454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.720611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.720692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.720851 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.720970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.721082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.721208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.721321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.721449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.721554 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.721702 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.721819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.721955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.722068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.722191 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.722322 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.722460 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.722573 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.722694 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.722858 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.722940 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.723102 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.723182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.723360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.723439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.723600 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.723684 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.723853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.723932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.724087 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] federation.caching = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.724166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.724320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.724401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.724562 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.724641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.724809 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.724889 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.725037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.725118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.725280 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.725360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.725516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.725596 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.725758 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.725838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.725989 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.726067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.726234 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.726380 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.726489 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.726570 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.726736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.726814 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.726969 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] identity.caching = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.727047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.727208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.727288 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.727449 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.727533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.727683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.727763 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.727912 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.727991 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.728150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.728230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.728377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.728455 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.728611 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.728691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.728846 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.728959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.729089 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.729211 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.729328 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.729442 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.729565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.729670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.729813 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.729892 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.730061 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.730143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.730311 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.730456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.730536 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.730704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.730794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.730955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.731034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.731199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.731278 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.731464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.731543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.731698 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.731776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.731927 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.732004 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.732154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.732235 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.732389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.732469 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.732618 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.732696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.732853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.732940 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.733092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.733178 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.733329 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.733408 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.733563 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.733644 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.733789 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.733926 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.734007 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.734067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.734248 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.734346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.734505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.734584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.734737 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.734816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.734985 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.735066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.735208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.735287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.735443 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.735522 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.735671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.735751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.735906 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.735986 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.736139 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.736219 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.736376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.736458 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.736612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.736691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.736845 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.736925 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.737078 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.737158 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.737313 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.737394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.737537 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.737616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.737764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.737845 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.738007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.738086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.738250 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.738354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.738507 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.738588 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.738739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.738816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.738972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.739051 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.739199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.739279 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.739434 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.739512 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.739682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.739761 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.739908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.739987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.740653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.740740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.740898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.740976 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.741139 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.741218 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] resource.caching = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.741371 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.741450 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.741604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.741685 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.741839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.741919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.742071 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.742150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.742326 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.742410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.742555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.742633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.742787 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.742867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.743027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.743105 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.743266 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.743345 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.743499 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] role.caching = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.743577 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.743738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.743817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] role.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.743965 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.744045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.744207 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.744286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.744438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.744517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.744676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.744757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.744909 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.744988 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.745140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.745228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.745382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.745461 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.745623 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.745702 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.745866 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.745953 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.746096 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.746174 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.746361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.746443 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.746607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.746698 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.746841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.746921 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.747075 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.747160 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.747307 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.747468 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.747548 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.747705 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.747784 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.747953 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.748033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.748203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.748284 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.748439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] resource.caching = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.748517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.748686 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.748766 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.748920 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.749000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.749152 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.749229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.749397 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.749476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.749643 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.749723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.749873 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.749953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.750107 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.750187 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.750409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.750502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.750598 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.750679 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.750830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] role.caching = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.750910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.751072 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] role.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.751155 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.751300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.751380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.751539 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.751619 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.751769 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.751850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.752006 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] token.caching = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.752086 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.752242 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.752322 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.752481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.752561 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.752715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.752794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.752953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.753040 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.753195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.753275 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.753422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.753502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.753663 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.753750 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.753903 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.753982 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.754145 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.754226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.754414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.754506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.754653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.754805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.754885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.755039 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.755118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.755286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.755368 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.755531 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.755611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.755778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.755857 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.756901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.756994 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.757134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.757220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.connection = **** {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.757388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.757479 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.757639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.757726 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.757874 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.757966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.758105 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.758185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.758358 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.758455 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.758591 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.758671 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.758825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.758904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.759061 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.759150 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.759305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.759385 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.759538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.759617 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.759772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.759861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.760000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.760079 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.760232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] token.caching = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.760310 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.760465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.760553 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.760706 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.760786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.760939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.761018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.761179 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.761271 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.761414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.761494 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.761568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 15:11:11.761641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.761872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.762112 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.762376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.762611 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.762847 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.763080 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.763322 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.763565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.763790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.764030 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.764317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.764573 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.connection = **** {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.764817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.765060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.765292 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.765529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.765762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.765998 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.766231 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.766488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.766735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.766968 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.767207 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.767440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.767683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.767915 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.768155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.768382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.768618 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.768849 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 15:11:11.769016 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 15:11:11.792087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:11.792320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:11.792548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:11.794565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:11.794798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:11.795029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:11.800953 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:11.803801 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:11.829709 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:11.829940 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:11.830039 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:11.830176 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:11.971663 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:11.971845 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.020778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.021013 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.021615 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.021850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.085701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.085929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.086151 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.086398 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.086621 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.086707 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.086839 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.086929 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.087100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.087317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.087530 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.087742 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.112305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.112580 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.112700 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.112817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.112904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.113041 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.113124 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.113263 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.113347 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.113478 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.113560 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.113693 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.113785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.113906 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.125344 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.125566 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.138184 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.138443 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.138545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.138674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.138758 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.138900 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.138983 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.139126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.139210 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.139340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.139423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.139556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.139688 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.139800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 15:11:12.156529 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.156736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 15:11:12.157074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.157158 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.157277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 15:11:12.157361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 15:11:12.157551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.157643 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.157814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.157906 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 15:11:12.158075 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.158157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.158360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.158446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.158618 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.158874 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.169943 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.170221 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.170408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.170518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.170683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.170801 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.170951 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.171292 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.179530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.180084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.182834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.183115 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.183371 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.183455 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.183645 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.183778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.183923 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.184065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.184314 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.184396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.184589 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.184671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.184846 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.184926 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.185113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.185204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.185386 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.185468 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.185655 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.185736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.185923 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.186188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.212302 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.212478 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.213908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.214072 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.214190 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.214370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.219475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.219602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.219762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.219879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.220044 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.220155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.227942 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.228038 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.228180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.228273 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.232792 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.232879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.233080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.233162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.233353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.233436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.233629 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.233711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.233896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.233986 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.234165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.234247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.234455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.234539 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.234780 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.234921 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.253776 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.253884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.254046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.254134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.258888 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.258989 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.262560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.262661 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.262852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.262948 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.267853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.267941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.271519 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.271613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.275323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.275431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.275616 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.275702 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.275905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.276004 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.276192 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.276282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.276476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.276576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.276749 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.276833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.296216 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.296312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.296588 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.296672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.296854 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.296944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.297120 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.297203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.305584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.305677 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.307314 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.307401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.307588 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.307686 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.310647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.310736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.310920 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.311003 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.311197 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.311294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.311474 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.311558 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.311753 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.311837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.325050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.325212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.325330 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.325455 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.325608 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.325736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.325885 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.326023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.326161 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.326325 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.326476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.326615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.326750 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.326890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.346346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.346558 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.350228 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.350459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.353534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.353761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.355161 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.355377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.356763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.356997 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.357081 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.357279 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.357371 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.357569 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.364670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.365106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.366337 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.366761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.369541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.369828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.369979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.370108 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.370288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.370406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.370581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.370867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.379560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.379842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.380096 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.380207 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.380382 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.380665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.388354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.388629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.388843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 15:11:12.388929 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.389123 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.389208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.389405 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.389489 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.389682 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.389766 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.389953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.390044 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.390223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.390325 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.390521 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.390605 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.390796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.390891 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.391068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.391150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.391327 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.391410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.391586 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.391670 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.391852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.391935 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.392115 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.392381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 15:11:12.430610 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556f5d3879c0 pid: 80508 (default app) Jul 20 15:11:12.431257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556f5d3879c0 pid: 80509 (default app) Jul 20 15:11:12.433880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c0f91-2f2f-42ca-b2b9-2a00dfbb4eb9 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:12.434170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c0f91-2f2f-42ca-b2b9-2a00dfbb4eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:12.434482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c0f91-2f2f-42ca-b2b9-2a00dfbb4eb9 None None] PATH_INFO: `/v3` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:12.438636 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1/1] 10.222.0.78 () {50 vars in 791 bytes} [Wed Jul 20 15:11:12 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:21.152782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df492d7f-9dc8-4e33-9890-c57a701df029 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:21.153285 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df492d7f-9dc8-4e33-9890-c57a701df029 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:21.153389 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df492d7f-9dc8-4e33-9890-c57a701df029 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:21.157639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1/2] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:11:21.162944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f81b9-9cca-4857-81a9-7f81c287cb02 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:21.163326 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f81b9-9cca-4857-81a9-7f81c287cb02 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:21.163991 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f81b9-9cca-4857-81a9-7f81c287cb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:21.233326 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: INFO dbcounter [None req-695f81b9-9cca-4857-81a9-7f81c287cb02 None None] Registered counter for database keystone Jul 20 15:11:21.254914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG oslo_db.sqlalchemy.engines [None req-695f81b9-9cca-4857-81a9-7f81c287cb02 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80508) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 15:11:21.261421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writer thread running {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 15:11:21.436730 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG passlib.handlers.bcrypt [None req-695f81b9-9cca-4857-81a9-7f81c287cb02 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80508) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 15:11:21.437062 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG passlib.handlers.bcrypt [None req-695f81b9-9cca-4857-81a9-7f81c287cb02 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80508) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 15:11:21.492711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-695f81b9-9cca-4857-81a9-7f81c287cb02 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:21.494349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-695f81b9-9cca-4857-81a9-7f81c287cb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:21.613332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 2/3] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 20 15:11:21.795368 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df492d7f-9dc8-4e33-9890-c57a701df029 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:21.865786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: INFO dbcounter [None req-df492d7f-9dc8-4e33-9890-c57a701df029 None None] Registered counter for database keystone Jul 20 15:11:21.885343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG oslo_db.sqlalchemy.engines [None req-df492d7f-9dc8-4e33-9890-c57a701df029 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80509) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 15:11:21.891488 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writer thread running {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 15:11:22.009925 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df492d7f-9dc8-4e33-9890-c57a701df029 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:22.014226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7945ce-779e-4d60-9b83-b5a3f4b346ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:22.014917 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7945ce-779e-4d60-9b83-b5a3f4b346ae admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:22.015205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7945ce-779e-4d60-9b83-b5a3f4b346ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:22.015487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7945ce-779e-4d60-9b83-b5a3f4b346ae admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:22.023745 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7945ce-779e-4d60-9b83-b5a3f4b346ae admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:22.075884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9f7945ce-779e-4d60-9b83-b5a3f4b346ae admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 15:11:22.076164 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7945ce-779e-4d60-9b83-b5a3f4b346ae admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:22.086136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 2/4] 10.222.0.78 () {58 vars in 1324 bytes} [Wed Jul 20 15:11:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:11:22.092244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f81b9-9cca-4857-81a9-7f81c287cb02 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:22.107101 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f81b9-9cca-4857-81a9-7f81c287cb02 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:22.111589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264f719-74ad-4628-bc6c-ecc1621283b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:22.112705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264f719-74ad-4628-bc6c-ecc1621283b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:22.112997 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264f719-74ad-4628-bc6c-ecc1621283b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:22.113277 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264f719-74ad-4628-bc6c-ecc1621283b4 admin admin] PATH_INFO: `/v3/projects/effa579779b9494f8b1dd5831edaae25` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:22.114815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b264f719-74ad-4628-bc6c-ecc1621283b4 admin admin] RBAC: Authorizing `identity:get_project(project_id=effa579779b9494f8b1dd5831edaae25)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:22.167949 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b264f719-74ad-4628-bc6c-ecc1621283b4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 15:11:22.168465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b264f719-74ad-4628-bc6c-ecc1621283b4 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:22.174383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 3/5] 10.222.0.78 () {56 vars in 1207 bytes} [Wed Jul 20 15:11:22 2022] GET /identity/v3/projects/effa579779b9494f8b1dd5831edaae25 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:24.168166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909797a-6804-4f2d-9214-8cf7eb7c0128 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:24.168643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909797a-6804-4f2d-9214-8cf7eb7c0128 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:24.168728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909797a-6804-4f2d-9214-8cf7eb7c0128 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:24.172817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 3/6] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:11:24.178151 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af57d9e-740e-4eca-9262-96ef5d101aae None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:24.178580 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af57d9e-740e-4eca-9262-96ef5d101aae None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:24.178808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af57d9e-740e-4eca-9262-96ef5d101aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:24.197848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-5af57d9e-740e-4eca-9262-96ef5d101aae None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:24.199441 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-5af57d9e-740e-4eca-9262-96ef5d101aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:24.207508 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 4/7] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 20 15:11:24.229894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9909797a-6804-4f2d-9214-8cf7eb7c0128 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:24.242490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9909797a-6804-4f2d-9214-8cf7eb7c0128 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:24.246702 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c77159-001d-440e-be9c-10d6c244d0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:24.247335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c77159-001d-440e-be9c-10d6c244d0ce admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:24.247621 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c77159-001d-440e-be9c-10d6c244d0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:24.247901 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c77159-001d-440e-be9c-10d6c244d0ce admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:24.255817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c77159-001d-440e-be9c-10d6c244d0ce admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:24.257576 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c77159-001d-440e-be9c-10d6c244d0ce admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:24.266244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 4/8] 10.222.0.78 () {58 vars in 1324 bytes} [Wed Jul 20 15:11:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:11:24.271482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af57d9e-740e-4eca-9262-96ef5d101aae None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:24.281007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af57d9e-740e-4eca-9262-96ef5d101aae None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:24.285382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4242147-83a1-4bae-a9be-02d4d54c1e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:24.286247 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4242147-83a1-4bae-a9be-02d4d54c1e93 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:24.286566 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4242147-83a1-4bae-a9be-02d4d54c1e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:24.286849 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4242147-83a1-4bae-a9be-02d4d54c1e93 admin admin] PATH_INFO: `/v3/users/6049ef02a0334b599d9582b3c2aa2ebf` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:24.288242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4242147-83a1-4bae-a9be-02d4d54c1e93 admin admin] RBAC: Authorizing `identity:get_user(user_id=6049ef02a0334b599d9582b3c2aa2ebf)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:24.290709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4242147-83a1-4bae-a9be-02d4d54c1e93 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:24.292272 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 5/9] 10.222.0.78 () {56 vars in 1198 bytes} [Wed Jul 20 15:11:24 2022] GET /identity/v3/users/6049ef02a0334b599d9582b3c2aa2ebf => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:26.380239 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c399119-aa36-4bf1-bb5b-3d56817b0a82 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:26.380998 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c399119-aa36-4bf1-bb5b-3d56817b0a82 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:26.380998 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c399119-aa36-4bf1-bb5b-3d56817b0a82 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:26.382447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 5/10] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:26.387651 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc70f2d-3511-49ca-813e-95e62bb18adc None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:26.388044 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc70f2d-3511-49ca-813e-95e62bb18adc None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:26.388333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc70f2d-3511-49ca-813e-95e62bb18adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:26.392199 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c646afb-016d-40b6-b4bf-1bde66e60c42 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:26.392492 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c646afb-016d-40b6-b4bf-1bde66e60c42 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:26.392776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c646afb-016d-40b6-b4bf-1bde66e60c42 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:26.394037 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 6/11] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:26.399167 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea7071-2105-432a-b598-1cc2fb40501d None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:26.399459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea7071-2105-432a-b598-1cc2fb40501d None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:26.399738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea7071-2105-432a-b598-1cc2fb40501d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:26.404992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-2bc70f2d-3511-49ca-813e-95e62bb18adc None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:26.406201 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2bc70f2d-3511-49ca-813e-95e62bb18adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:26.439251 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG passlib.handlers.bcrypt [None req-93ea7071-2105-432a-b598-1cc2fb40501d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80509) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 15:11:26.439664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG passlib.handlers.bcrypt [None req-93ea7071-2105-432a-b598-1cc2fb40501d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80509) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 15:11:26.485083 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-93ea7071-2105-432a-b598-1cc2fb40501d None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:26.486895 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 6/12] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:26.487008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-93ea7071-2105-432a-b598-1cc2fb40501d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:26.510684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc70f2d-3511-49ca-813e-95e62bb18adc None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:26.534727 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:26.589295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 7/13] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:11:26.611820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea7071-2105-432a-b598-1cc2fb40501d None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:26.616104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc70f2d-3511-49ca-813e-95e62bb18adc None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:26.649836 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f874be-a9b0-4844-bf95-bd365ab145b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:26.650595 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f874be-a9b0-4844-bf95-bd365ab145b9 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:26.650880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f874be-a9b0-4844-bf95-bd365ab145b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:26.651159 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f874be-a9b0-4844-bf95-bd365ab145b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:26.662377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f874be-a9b0-4844-bf95-bd365ab145b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:26.674070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-b9f874be-a9b0-4844-bf95-bd365ab145b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:11:26.674070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:26.674070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:26.674070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:26.674070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:26.674070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:26.674070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:26.674070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:26.674070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:26.674070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:26.674070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:26.674070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:26.674070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:26.674070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:26.674070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:26.674070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:26.675213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:26.675213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:26.675213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:26.675213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:26.675213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:26.675213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:26.675213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:26.675213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:26.675213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:26.675213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:26.675213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:26.675213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:26.675213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:26.675213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:26.675213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:26.675213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:26.676167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:26.676167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:26.676167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:26.676167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:26.676167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:26.676167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:26.676167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:26.676167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:26.676167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:26.676167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:26.676167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:11:26.676167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:11:26.676167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 7/14] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:11:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:11:26.681124 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f874be-a9b0-4844-bf95-bd365ab145b9 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:26.712177 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea7071-2105-432a-b598-1cc2fb40501d None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:26.714444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:26.748177 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be881b-c1bf-411d-9af4-a0d68a984142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:26.751105 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be881b-c1bf-411d-9af4-a0d68a984142 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:26.751105 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be881b-c1bf-411d-9af4-a0d68a984142 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:26.751105 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be881b-c1bf-411d-9af4-a0d68a984142 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:26.760907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:26.779513 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f874be-a9b0-4844-bf95-bd365ab145b9 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:26.814496 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826c712-dba3-4ad4-9d17-4664200e6acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:26.815472 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826c712-dba3-4ad4-9d17-4664200e6acf None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:26.815757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826c712-dba3-4ad4-9d17-4664200e6acf None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:26.816037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826c712-dba3-4ad4-9d17-4664200e6acf None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:26.816799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826c712-dba3-4ad4-9d17-4664200e6acf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:26.817106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826c712-dba3-4ad4-9d17-4664200e6acf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:26.818449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be881b-c1bf-411d-9af4-a0d68a984142 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:26.818569 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826c712-dba3-4ad4-9d17-4664200e6acf None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:26.819995 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be881b-c1bf-411d-9af4-a0d68a984142 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:26.831658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 8/15] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:11:26.837756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826c712-dba3-4ad4-9d17-4664200e6acf None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:26.918792 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 8/16] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:11:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 15:11:26.924466 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be881b-c1bf-411d-9af4-a0d68a984142 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:26.928234 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:26.936522 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826c712-dba3-4ad4-9d17-4664200e6acf None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:26.971187 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df0d5b-50fe-4917-9357-11ade814c39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:26.972070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df0d5b-50fe-4917-9357-11ade814c39e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:26.972351 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df0d5b-50fe-4917-9357-11ade814c39e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:26.972634 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df0d5b-50fe-4917-9357-11ade814c39e None admin] PATH_INFO: `/v3/users/6049ef02a0334b599d9582b3c2aa2ebf` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:26.973851 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df0d5b-50fe-4917-9357-11ade814c39e None admin] RBAC: Authorizing `identity:get_user(user_id=6049ef02a0334b599d9582b3c2aa2ebf)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:26.975319 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df0d5b-50fe-4917-9357-11ade814c39e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:26.976956 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 9/17] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:26 2022] GET /identity/v3/users/6049ef02a0334b599d9582b3c2aa2ebf => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:11:26.982355 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df0d5b-50fe-4917-9357-11ade814c39e None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:26.995990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:27.024762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be881b-c1bf-411d-9af4-a0d68a984142 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:27.059097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48989d6-e750-4ec4-ace8-4cc480ee0dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:27.059887 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48989d6-e750-4ec4-ace8-4cc480ee0dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:27.060172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48989d6-e750-4ec4-ace8-4cc480ee0dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:27.060451 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48989d6-e750-4ec4-ace8-4cc480ee0dd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:27.079135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df0d5b-50fe-4917-9357-11ade814c39e None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:27.079291 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48989d6-e750-4ec4-ace8-4cc480ee0dd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:27.080658 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48989d6-e750-4ec4-ace8-4cc480ee0dd8 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:27.104581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-a48989d6-e750-4ec4-ace8-4cc480ee0dd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:11:27.104581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:27.104581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:27.104581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:27.104581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:27.104581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:27.104581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:27.104581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:27.104581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:27.104581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:27.104581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:27.104581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:27.104581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:11:27.104581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:11:27.104581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:11:27.105587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:11:27.105587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:27.105587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:27.105587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:27.105587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:27.105587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:27.105587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:27.105587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:27.105587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:27.105587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:27.105587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:27.105587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:27.105587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:27.105587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:27.105587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:27.105587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:27.105587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:27.106450 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:11:27.106450 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:11:27.106450 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:11:27.106450 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:11:27.106450 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 9/18] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:11:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:11:27.111734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48989d6-e750-4ec4-ace8-4cc480ee0dd8 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:27.117042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd3bef-c2b4-4fd3-aac0-82cc7a658ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:27.117843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd3bef-c2b4-4fd3-aac0-82cc7a658ffd None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:27.118128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd3bef-c2b4-4fd3-aac0-82cc7a658ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:27.118438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd3bef-c2b4-4fd3-aac0-82cc7a658ffd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:27.119627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd3bef-c2b4-4fd3-aac0-82cc7a658ffd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:27.121084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd3bef-c2b4-4fd3-aac0-82cc7a658ffd None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:27.122892 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 10/19] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:11:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:27.128383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd3bef-c2b4-4fd3-aac0-82cc7a658ffd None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:27.169805 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:27.214005 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48989d6-e750-4ec4-ace8-4cc480ee0dd8 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:27.215838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:27.235606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd3bef-c2b4-4fd3-aac0-82cc7a658ffd None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:27.247275 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dcb81-dd76-4db6-b57b-09955c6bd3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:27.247949 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dcb81-dd76-4db6-b57b-09955c6bd3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:27.248237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dcb81-dd76-4db6-b57b-09955c6bd3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:27.248514 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dcb81-dd76-4db6-b57b-09955c6bd3e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:27.249272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dcb81-dd76-4db6-b57b-09955c6bd3e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:27.249578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dcb81-dd76-4db6-b57b-09955c6bd3e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:27.251011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dcb81-dd76-4db6-b57b-09955c6bd3e1 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:27.268847 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 10/20] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:11:27.271578 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea836c24-cf57-48fd-9a15-67dfc556bd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:27.272250 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea836c24-cf57-48fd-9a15-67dfc556bd7b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:27.272533 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea836c24-cf57-48fd-9a15-67dfc556bd7b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:27.272858 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea836c24-cf57-48fd-9a15-67dfc556bd7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:27.273779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea836c24-cf57-48fd-9a15-67dfc556bd7b None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.domain.id=default, user.id=6049ef02a0334b599d9582b3c2aa2ebf) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:27.274092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea836c24-cf57-48fd-9a15-67dfc556bd7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:27.275650 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea836c24-cf57-48fd-9a15-67dfc556bd7b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:27.283400 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 11/21] 10.222.0.78 () {56 vars in 1322 bytes} [Wed Jul 20 15:11:27 2022] GET /identity/v3/role_assignments?user.id=6049ef02a0334b599d9582b3c2aa2ebf&scope.domain.id=default&role.id=25516be1485c49538544ea8a596e7d17 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:11:29.515005 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb93a8-f4b0-45e7-b017-0cd0d76323e2 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:29.515486 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb93a8-f4b0-45e7-b017-0cd0d76323e2 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:29.515586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb93a8-f4b0-45e7-b017-0cd0d76323e2 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:29.517205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 11/22] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:29.523117 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197492f1-f618-46bb-b7f2-b3697d5916f4 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:29.523517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197492f1-f618-46bb-b7f2-b3697d5916f4 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:29.523693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197492f1-f618-46bb-b7f2-b3697d5916f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:29.551297 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-197492f1-f618-46bb-b7f2-b3697d5916f4 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:29.552521 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-197492f1-f618-46bb-b7f2-b3697d5916f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:29.640390 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 12/23] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:11:29.678594 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb93a8-f4b0-45e7-b017-0cd0d76323e2 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:29.691922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d2707-5dec-49ff-af61-746b66ef1689 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:29.692229 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d2707-5dec-49ff-af61-746b66ef1689 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:29.692495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d2707-5dec-49ff-af61-746b66ef1689 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:29.694159 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 13/24] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:29.703103 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f46993-f449-4903-a1df-b65792b24c59 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:29.703648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f46993-f449-4903-a1df-b65792b24c59 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:29.704140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f46993-f449-4903-a1df-b65792b24c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:29.739322 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-00f46993-f449-4903-a1df-b65792b24c59 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:29.740903 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-00f46993-f449-4903-a1df-b65792b24c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:29.839765 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb93a8-f4b0-45e7-b017-0cd0d76323e2 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:29.874221 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:29.911978 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662c3f4-fa72-4d93-bb46-681f6f177030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:29.912789 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 14/25] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:11:29.912907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662c3f4-fa72-4d93-bb46-681f6f177030 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:29.913194 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662c3f4-fa72-4d93-bb46-681f6f177030 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:29.913483 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662c3f4-fa72-4d93-bb46-681f6f177030 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:29.940672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662c3f4-fa72-4d93-bb46-681f6f177030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:29.947437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f46993-f449-4903-a1df-b65792b24c59 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:29.952928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-8662c3f4-fa72-4d93-bb46-681f6f177030 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:11:29.952928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:29.952928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:29.952928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:29.952928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:29.952928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:29.952928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:29.952928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:29.952928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:29.952928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:29.952928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:29.952928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:29.952928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:29.952928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:29.952928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:29.952928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:29.953777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:29.953777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:29.953777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:29.953777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:29.953777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:29.953777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:29.953777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:29.953777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:29.953777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:29.953777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:29.953777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:29.953777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:29.953777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:29.953777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:29.953777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:29.953777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:29.954620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:29.954620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:29.954620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:29.954620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:29.954620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:29.954620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:29.954620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:29.954620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:29.954620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:29.954620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:29.954620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:11:29.954620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:11:29.954620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 12/26] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:11:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:29.956104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:29.967504 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662c3f4-fa72-4d93-bb46-681f6f177030 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:30.154184 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662c3f4-fa72-4d93-bb46-681f6f177030 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:30.222325 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f46993-f449-4903-a1df-b65792b24c59 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:30.223188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd9d0b-7ea4-4e72-97a1-6be2cf166368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:30.224315 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd9d0b-7ea4-4e72-97a1-6be2cf166368 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:30.224799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd9d0b-7ea4-4e72-97a1-6be2cf166368 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:30.225286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd9d0b-7ea4-4e72-97a1-6be2cf166368 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:30.226332 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd9d0b-7ea4-4e72-97a1-6be2cf166368 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:30.226848 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd9d0b-7ea4-4e72-97a1-6be2cf166368 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:30.228528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd9d0b-7ea4-4e72-97a1-6be2cf166368 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:30.252225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 13/27] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:11:30.261550 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd9d0b-7ea4-4e72-97a1-6be2cf166368 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:30.302481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce2109-24ce-4b04-a91c-59c46d4b82e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:30.304374 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce2109-24ce-4b04-a91c-59c46d4b82e4 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:30.305659 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce2109-24ce-4b04-a91c-59c46d4b82e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:30.306954 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce2109-24ce-4b04-a91c-59c46d4b82e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:30.323156 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce2109-24ce-4b04-a91c-59c46d4b82e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:30.324976 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce2109-24ce-4b04-a91c-59c46d4b82e4 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:30.332714 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:30.346005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-f8ce2109-24ce-4b04-a91c-59c46d4b82e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:11:30.346005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:30.346005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:30.346005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:30.346005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:30.346005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:30.346005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:30.346005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:30.346005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:30.346005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:30.346005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:30.346005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:30.346005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:11:30.346005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:11:30.346005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:11:30.349757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:11:30.349757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:30.349757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:30.349757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:30.349757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:30.349757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:30.349757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:30.349757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:30.349757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:30.349757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:30.349757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:30.349757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:30.349757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:30.349757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:30.349757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:30.349757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:30.349757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:30.350709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:11:30.350709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:11:30.350709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:11:30.350709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:11:30.350709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 15/28] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:11:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:30.350709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8a207-46a4-43cb-aa79-b1c8741e573c None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:30.351618 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8a207-46a4-43cb-aa79-b1c8741e573c None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:30.354571 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8a207-46a4-43cb-aa79-b1c8741e573c None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:30.356225 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 16/29] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:30.357166 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8a207-46a4-43cb-aa79-b1c8741e573c None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:30.420510 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd9d0b-7ea4-4e72-97a1-6be2cf166368 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:30.427166 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:30.458487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e60d3d-b1c3-455b-8978-cc6dcc0e6119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:30.459372 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e60d3d-b1c3-455b-8978-cc6dcc0e6119 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:30.459655 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e60d3d-b1c3-455b-8978-cc6dcc0e6119 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:30.460004 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e60d3d-b1c3-455b-8978-cc6dcc0e6119 None admin] PATH_INFO: `/v3/users/6049ef02a0334b599d9582b3c2aa2ebf` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:30.461927 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e60d3d-b1c3-455b-8978-cc6dcc0e6119 None admin] RBAC: Authorizing `identity:get_user(user_id=6049ef02a0334b599d9582b3c2aa2ebf)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:30.463295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e60d3d-b1c3-455b-8978-cc6dcc0e6119 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:30.465118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 14/30] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:30 2022] GET /identity/v3/users/6049ef02a0334b599d9582b3c2aa2ebf => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:30.466231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac28d90-6c24-4745-90f2-7842a2937bff None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:30.466544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac28d90-6c24-4745-90f2-7842a2937bff None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:30.466821 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac28d90-6c24-4745-90f2-7842a2937bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:30.480464 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8a207-46a4-43cb-aa79-b1c8741e573c None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:30.484612 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-7ac28d90-6c24-4745-90f2-7842a2937bff None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:30.485818 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-7ac28d90-6c24-4745-90f2-7842a2937bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:30.518576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22759c4a-624c-4e54-8ad1-f9b2aca301c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:30.519594 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22759c4a-624c-4e54-8ad1-f9b2aca301c2 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:30.520170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22759c4a-624c-4e54-8ad1-f9b2aca301c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:30.520744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22759c4a-624c-4e54-8ad1-f9b2aca301c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:30.521850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22759c4a-624c-4e54-8ad1-f9b2aca301c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:30.522377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22759c4a-624c-4e54-8ad1-f9b2aca301c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:30.524231 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22759c4a-624c-4e54-8ad1-f9b2aca301c2 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:30.543272 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 17/31] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:30.544931 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70209d-cc75-44be-8cd2-f1deebc57a51 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:30.545505 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70209d-cc75-44be-8cd2-f1deebc57a51 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:30.546078 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70209d-cc75-44be-8cd2-f1deebc57a51 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:30.547841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 18/32] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:30.548844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70209d-cc75-44be-8cd2-f1deebc57a51 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:30.562302 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 15/33] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:30.571171 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac28d90-6c24-4745-90f2-7842a2937bff None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:30.623262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:30.659639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70209d-cc75-44be-8cd2-f1deebc57a51 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:30.676937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:30.695265 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac28d90-6c24-4745-90f2-7842a2937bff None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:30.695875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedb8df-039a-494d-8ae1-e6b3ef0ba79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:30.696883 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedb8df-039a-494d-8ae1-e6b3ef0ba79b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:30.697375 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedb8df-039a-494d-8ae1-e6b3ef0ba79b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:30.697864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedb8df-039a-494d-8ae1-e6b3ef0ba79b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:30.699324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedb8df-039a-494d-8ae1-e6b3ef0ba79b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:30.700979 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedb8df-039a-494d-8ae1-e6b3ef0ba79b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:30.703151 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 19/34] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:11:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:30.704561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad611f3-fb9e-4d23-b64e-293bed48e250 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:30.705068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad611f3-fb9e-4d23-b64e-293bed48e250 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:30.705557 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad611f3-fb9e-4d23-b64e-293bed48e250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:30.722567 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-2ad611f3-fb9e-4d23-b64e-293bed48e250 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:30.724041 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2ad611f3-fb9e-4d23-b64e-293bed48e250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:30.733972 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2ae61-f6af-46af-86b2-34d26d37104a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:30.734702 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2ae61-f6af-46af-86b2-34d26d37104a None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:30.734990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2ae61-f6af-46af-86b2-34d26d37104a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:30.735326 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2ae61-f6af-46af-86b2-34d26d37104a None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:30.736212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2ae61-f6af-46af-86b2-34d26d37104a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:30.737892 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2ae61-f6af-46af-86b2-34d26d37104a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:30.796084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 20/35] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:30.797428 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad611f3-fb9e-4d23-b64e-293bed48e250 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:30.909648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad611f3-fb9e-4d23-b64e-293bed48e250 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:30.925424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:30.946131 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e5565-46af-4674-9da5-74caa1de86fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:30.947209 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e5565-46af-4674-9da5-74caa1de86fd None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:30.947705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e5565-46af-4674-9da5-74caa1de86fd None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:30.948216 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e5565-46af-4674-9da5-74caa1de86fd None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:30.949150 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e5565-46af-4674-9da5-74caa1de86fd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:30.950789 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e5565-46af-4674-9da5-74caa1de86fd None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:30.974681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 16/36] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Jul 20 15:11:30 2022] POST /identity/v3/projects => generated 301 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:11:30.975530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2ae61-f6af-46af-86b2-34d26d37104a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:31.090838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:31.108198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2ae61-f6af-46af-86b2-34d26d37104a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:31.159249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d0884-138b-4235-b8f7-b566399b0f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:31.160228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d0884-138b-4235-b8f7-b566399b0f39 None admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:31.160465 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d0884-138b-4235-b8f7-b566399b0f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:31.160753 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d0884-138b-4235-b8f7-b566399b0f39 None admin] PATH_INFO: `/v3/domains/default/users/6049ef02a0334b599d9582b3c2aa2ebf/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:31.163241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d0884-138b-4235-b8f7-b566399b0f39 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6049ef02a0334b599d9582b3c2aa2ebf, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:31.164862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d0884-138b-4235-b8f7-b566399b0f39 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:31.292505 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 21/37] 10.222.0.78 () {60 vars in 1131 bytes} [Wed Jul 20 15:11:30 2022] POST /identity/v3/roles => generated 219 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:11:31.293463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e5565-46af-4674-9da5-74caa1de86fd None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:31.327011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 17/38] 10.222.0.78 () {58 vars in 1361 bytes} [Wed Jul 20 15:11:30 2022] PUT /identity/v3/domains/default/users/6049ef02a0334b599d9582b3c2aa2ebf/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:11:31.328938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bf363-682a-43a3-bcdf-e7a508ebca7a None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:31.329490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bf363-682a-43a3-bcdf-e7a508ebca7a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:31.329999 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bf363-682a-43a3-bcdf-e7a508ebca7a None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:31.332133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 18/39] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:31.340576 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff0373-4167-40ab-aad2-7eb610fe6148 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:31.341587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff0373-4167-40ab-aad2-7eb610fe6148 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:31.342599 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff0373-4167-40ab-aad2-7eb610fe6148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:31.371961 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-70ff0373-4167-40ab-aad2-7eb610fe6148 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:31.374127 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-70ff0373-4167-40ab-aad2-7eb610fe6148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:31.451874 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e5565-46af-4674-9da5-74caa1de86fd None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:31.462507 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 19/40] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:31.464356 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990d75d-a5ba-41dd-8b68-37a754737f31 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:31.464893 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990d75d-a5ba-41dd-8b68-37a754737f31 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:31.465478 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990d75d-a5ba-41dd-8b68-37a754737f31 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:31.467139 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 20/41] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:31.474865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5944a2-deaf-43bd-be62-f942845d0a7e None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:31.475466 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5944a2-deaf-43bd-be62-f942845d0a7e None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:31.476030 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5944a2-deaf-43bd-be62-f942845d0a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:31.495248 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-3b5944a2-deaf-43bd-be62-f942845d0a7e None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:31.496786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-3b5944a2-deaf-43bd-be62-f942845d0a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:31.497479 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc2cc3-a06c-4538-bff0-a72ba9dd5064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:31.498221 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc2cc3-a06c-4538-bff0-a72ba9dd5064 None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:31.498556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc2cc3-a06c-4538-bff0-a72ba9dd5064 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:31.498841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc2cc3-a06c-4538-bff0-a72ba9dd5064 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:31.499661 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc2cc3-a06c-4538-bff0-a72ba9dd5064 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:31.501883 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc2cc3-a06c-4538-bff0-a72ba9dd5064 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:31.572436 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 22/42] 10.222.0.78 () {60 vars in 1131 bytes} [Wed Jul 20 15:11:31 2022] POST /identity/v3/roles => generated 225 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:11:31.573343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc2cc3-a06c-4538-bff0-a72ba9dd5064 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:31.574437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 21/43] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:11:31.606060 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5944a2-deaf-43bd-be62-f942845d0a7e None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:31.622769 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:31.627253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:31.712478 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc2cc3-a06c-4538-bff0-a72ba9dd5064 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:31.749241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5944a2-deaf-43bd-be62-f942845d0a7e None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:31.756048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827eb82-2a8d-40d0-94cb-7885b774aacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:31.757604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827eb82-2a8d-40d0-94cb-7885b774aacb None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:31.758217 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827eb82-2a8d-40d0-94cb-7885b774aacb None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:31.758815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827eb82-2a8d-40d0-94cb-7885b774aacb None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:31.763132 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827eb82-2a8d-40d0-94cb-7885b774aacb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:31.764882 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827eb82-2a8d-40d0-94cb-7885b774aacb None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:31.791079 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db475c6-ac09-4ff3-853a-e0193c5d1c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:31.792176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db475c6-ac09-4ff3-853a-e0193c5d1c63 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:31.792668 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db475c6-ac09-4ff3-853a-e0193c5d1c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:31.793129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db475c6-ac09-4ff3-853a-e0193c5d1c63 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:31.794579 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db475c6-ac09-4ff3-853a-e0193c5d1c63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:31.796136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db475c6-ac09-4ff3-853a-e0193c5d1c63 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:31.797775 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 22/44] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:11:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:11:31.803257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db475c6-ac09-4ff3-853a-e0193c5d1c63 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:31.822699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 23/45] 10.222.0.78 () {60 vars in 1131 bytes} [Wed Jul 20 15:11:31 2022] POST /identity/v3/roles => generated 223 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:11:31.895830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:31.920743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db475c6-ac09-4ff3-853a-e0193c5d1c63 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:31.957933 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d622d-d8c9-484f-876a-f3827e380e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:31.958661 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d622d-d8c9-484f-876a-f3827e380e34 None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:31.958954 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d622d-d8c9-484f-876a-f3827e380e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:31.959294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d622d-d8c9-484f-876a-f3827e380e34 None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:31.960070 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d622d-d8c9-484f-876a-f3827e380e34 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:31.961560 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d622d-d8c9-484f-876a-f3827e380e34 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:32.068408 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 23/46] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Jul 20 15:11:31 2022] POST /identity/v3/projects => generated 312 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:11:33.426587 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3feee-b847-4851-988a-327eb31cf476 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:33.427084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3feee-b847-4851-988a-327eb31cf476 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:33.427084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3feee-b847-4851-988a-327eb31cf476 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:33.428454 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 24/47] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:33.434217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac14e24-adac-423c-b9aa-79622c667b08 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:33.434623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac14e24-adac-423c-b9aa-79622c667b08 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:33.434937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac14e24-adac-423c-b9aa-79622c667b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:33.452576 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-eac14e24-adac-423c-b9aa-79622c667b08 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:33.453760 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-eac14e24-adac-423c-b9aa-79622c667b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:33.623968 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 24/48] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:33.647255 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3feee-b847-4851-988a-327eb31cf476 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:33.740561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:33.748070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3feee-b847-4851-988a-327eb31cf476 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:33.782025 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a489bfd-08da-4892-a8c4-07b07087f562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:33.782866 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a489bfd-08da-4892-a8c4-07b07087f562 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:33.783160 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a489bfd-08da-4892-a8c4-07b07087f562 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:33.783440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a489bfd-08da-4892-a8c4-07b07087f562 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:33.793449 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a489bfd-08da-4892-a8c4-07b07087f562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:33.801170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-4a489bfd-08da-4892-a8c4-07b07087f562 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:11:33.801170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:33.801170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:33.801170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:33.801170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:33.801170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:33.801170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:33.801170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:33.801170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:33.801170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:33.801170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:33.801170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:33.801170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:33.801170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:33.801170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:33.801170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:33.802212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:33.802212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:33.802212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:33.802212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:33.802212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:33.802212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:33.802212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:33.802212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:33.802212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:33.802212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:33.802212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:33.802212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:33.802212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:33.802212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:33.802212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:33.802212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:33.803042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:33.803042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:33.803042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:33.803042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:33.803042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:33.803042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:33.803042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:33.803042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:33.803042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:33.803042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:33.803042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:11:33.803042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:11:33.803042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 25/49] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:11:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:33.809787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac14e24-adac-423c-b9aa-79622c667b08 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:33.873486 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:33.915113 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac14e24-adac-423c-b9aa-79622c667b08 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:33.947878 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a87e9-0228-44cd-a460-99b95c638d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:33.948549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a87e9-0228-44cd-a460-99b95c638d2d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:33.948838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a87e9-0228-44cd-a460-99b95c638d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:33.949121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a87e9-0228-44cd-a460-99b95c638d2d None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:33.949877 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a87e9-0228-44cd-a460-99b95c638d2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:33.950189 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a87e9-0228-44cd-a460-99b95c638d2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:33.951607 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a87e9-0228-44cd-a460-99b95c638d2d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:33.960706 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 25/50] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:33.966476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a489bfd-08da-4892-a8c4-07b07087f562 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:34.062498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a489bfd-08da-4892-a8c4-07b07087f562 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:34.097623 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446af664-48e2-4f8e-a837-4d79fac4454e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:34.098549 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446af664-48e2-4f8e-a837-4d79fac4454e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:34.098838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446af664-48e2-4f8e-a837-4d79fac4454e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:34.099118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446af664-48e2-4f8e-a837-4d79fac4454e None admin] PATH_INFO: `/v3/users/6049ef02a0334b599d9582b3c2aa2ebf` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:34.100320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446af664-48e2-4f8e-a837-4d79fac4454e None admin] RBAC: Authorizing `identity:get_user(user_id=6049ef02a0334b599d9582b3c2aa2ebf)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:34.101832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446af664-48e2-4f8e-a837-4d79fac4454e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:34.103513 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 26/51] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:33 2022] GET /identity/v3/users/6049ef02a0334b599d9582b3c2aa2ebf => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:34.108888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a87e9-0228-44cd-a460-99b95c638d2d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:34.112939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de8f10-1227-4ad8-8deb-2857dfbbad43 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:34.113233 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de8f10-1227-4ad8-8deb-2857dfbbad43 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:34.113518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de8f10-1227-4ad8-8deb-2857dfbbad43 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:34.114884 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 27/52] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:11:34.120113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135163f-7520-46e7-a91a-fcef0afe25db None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:34.120406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135163f-7520-46e7-a91a-fcef0afe25db None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:34.120688 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135163f-7520-46e7-a91a-fcef0afe25db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:34.137999 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-9135163f-7520-46e7-a91a-fcef0afe25db None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:34.139554 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9135163f-7520-46e7-a91a-fcef0afe25db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:34.155385 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:34.204492 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 28/53] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:11:34.211269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a87e9-0228-44cd-a460-99b95c638d2d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:34.227021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135163f-7520-46e7-a91a-fcef0afe25db None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:34.230620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:34.245013 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d94e1f-498b-4248-8d2b-f58ea0e3366a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:34.245806 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d94e1f-498b-4248-8d2b-f58ea0e3366a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:34.246087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d94e1f-498b-4248-8d2b-f58ea0e3366a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:34.246393 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d94e1f-498b-4248-8d2b-f58ea0e3366a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:34.247568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d94e1f-498b-4248-8d2b-f58ea0e3366a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:34.249149 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d94e1f-498b-4248-8d2b-f58ea0e3366a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:34.250881 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 26/54] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:11:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:34.256720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d94e1f-498b-4248-8d2b-f58ea0e3366a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:34.326951 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135163f-7520-46e7-a91a-fcef0afe25db None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:34.360708 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bf54f-d35d-4c39-b163-c37810eaeea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:34.361464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d94e1f-498b-4248-8d2b-f58ea0e3366a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:34.361569 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bf54f-d35d-4c39-b163-c37810eaeea4 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:34.361773 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bf54f-d35d-4c39-b163-c37810eaeea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:34.362050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bf54f-d35d-4c39-b163-c37810eaeea4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:34.363332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bf54f-d35d-4c39-b163-c37810eaeea4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:34.364847 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bf54f-d35d-4c39-b163-c37810eaeea4 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:34.367233 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 29/55] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:11:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:11:34.372824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bf54f-d35d-4c39-b163-c37810eaeea4 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:34.394639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:34.396831 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7472a0f-e0d4-46d7-b6f1-d18987cca813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:34.397524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7472a0f-e0d4-46d7-b6f1-d18987cca813 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:34.397816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7472a0f-e0d4-46d7-b6f1-d18987cca813 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:34.398105 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7472a0f-e0d4-46d7-b6f1-d18987cca813 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:34.398979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7472a0f-e0d4-46d7-b6f1-d18987cca813 None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.domain.id=default, user.id=6049ef02a0334b599d9582b3c2aa2ebf) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:34.399299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7472a0f-e0d4-46d7-b6f1-d18987cca813 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:34.400759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7472a0f-e0d4-46d7-b6f1-d18987cca813 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:34.408941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 27/56] 10.222.0.78 () {56 vars in 1322 bytes} [Wed Jul 20 15:11:34 2022] GET /identity/v3/role_assignments?user.id=6049ef02a0334b599d9582b3c2aa2ebf&scope.domain.id=default&role.id=25516be1485c49538544ea8a596e7d17 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:11:34.477049 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bf54f-d35d-4c39-b163-c37810eaeea4 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:34.514529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988682f-1b13-4c9a-ad99-04bace1adab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:34.515242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988682f-1b13-4c9a-ad99-04bace1adab9 None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:34.515532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988682f-1b13-4c9a-ad99-04bace1adab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:34.515814 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988682f-1b13-4c9a-ad99-04bace1adab9 None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:34.516555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988682f-1b13-4c9a-ad99-04bace1adab9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:34.518145 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988682f-1b13-4c9a-ad99-04bace1adab9 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:34.632820 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 30/57] 10.222.0.78 () {60 vars in 1140 bytes} [Wed Jul 20 15:11:34 2022] POST /identity/v3/projects => generated 298 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:11:36.636197 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137de2c-32e4-4ae8-b133-05e0079af683 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:36.636700 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137de2c-32e4-4ae8-b133-05e0079af683 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:36.636806 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137de2c-32e4-4ae8-b133-05e0079af683 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:36.638119 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 28/58] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:36.643317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b2ad7-d453-404f-8af8-42cee3143990 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:36.643688 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b2ad7-d453-404f-8af8-42cee3143990 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:36.643885 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b2ad7-d453-404f-8af8-42cee3143990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:36.660793 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-847b2ad7-d453-404f-8af8-42cee3143990 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:36.661983 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-847b2ad7-d453-404f-8af8-42cee3143990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:36.725133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 31/59] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:36.748944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137de2c-32e4-4ae8-b133-05e0079af683 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:36.802383 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:36.848774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137de2c-32e4-4ae8-b133-05e0079af683 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:36.880530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfdef2-00bf-41b6-ac95-46a420848776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:36.881307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfdef2-00bf-41b6-ac95-46a420848776 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:36.882099 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfdef2-00bf-41b6-ac95-46a420848776 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:36.882418 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfdef2-00bf-41b6-ac95-46a420848776 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:36.883611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cfdef2-00bf-41b6-ac95-46a420848776 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:36.885069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cfdef2-00bf-41b6-ac95-46a420848776 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:36.886752 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 29/60] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:11:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:36.892096 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b2ad7-d453-404f-8af8-42cee3143990 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:36.894285 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:36.999173 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b2ad7-d453-404f-8af8-42cee3143990 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:37.031479 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fcb6e-6b67-42b9-9982-336d42071fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:37.032144 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fcb6e-6b67-42b9-9982-336d42071fe7 None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:37.032428 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fcb6e-6b67-42b9-9982-336d42071fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:37.032703 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fcb6e-6b67-42b9-9982-336d42071fe7 None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:37.033406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fcb6e-6b67-42b9-9982-336d42071fe7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:37.034900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fcb6e-6b67-42b9-9982-336d42071fe7 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:37.158325 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 32/61] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:11:36 2022] POST /identity/v3/users => generated 275 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:11:40.304534 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d821c6-c16a-48d8-9dcd-260def7313ce None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:40.305031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d821c6-c16a-48d8-9dcd-260def7313ce None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:40.305104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d821c6-c16a-48d8-9dcd-260def7313ce None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:40.306766 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 30/62] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:40.313006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0aad6f-4e5c-4f2f-b7e2-213e3c2aeade None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:40.313510 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0aad6f-4e5c-4f2f-b7e2-213e3c2aeade None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:40.313587 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0aad6f-4e5c-4f2f-b7e2-213e3c2aeade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:40.316896 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab29d4d-6ebe-4a25-98e9-2983d1e1a0aa None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:40.317184 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab29d4d-6ebe-4a25-98e9-2983d1e1a0aa None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:40.317466 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab29d4d-6ebe-4a25-98e9-2983d1e1a0aa None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:40.318819 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 31/63] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:11:40.329105 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f11962-5e6d-4ee2-80ee-b964f189c42f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:40.329640 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f11962-5e6d-4ee2-80ee-b964f189c42f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:40.330119 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f11962-5e6d-4ee2-80ee-b964f189c42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:40.355937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-0a0aad6f-4e5c-4f2f-b7e2-213e3c2aeade None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:40.357504 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0a0aad6f-4e5c-4f2f-b7e2-213e3c2aeade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:40.358789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-c6f11962-5e6d-4ee2-80ee-b964f189c42f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:40.363516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-c6f11962-5e6d-4ee2-80ee-b964f189c42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:40.458418 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:40.479816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 32/64] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:11:40.486283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 33/65] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:40.507814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f11962-5e6d-4ee2-80ee-b964f189c42f None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:40.513251 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa27f05-daf9-45f1-9a53-67ceb53db8e5 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:40.513962 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa27f05-daf9-45f1-9a53-67ceb53db8e5 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:40.514517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa27f05-daf9-45f1-9a53-67ceb53db8e5 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:40.516242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 34/66] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:40.524149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9028b-c4a7-46ed-b994-2819b1b5d808 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:40.524443 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9028b-c4a7-46ed-b994-2819b1b5d808 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:40.524723 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9028b-c4a7-46ed-b994-2819b1b5d808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:40.543564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-50f9028b-c4a7-46ed-b994-2819b1b5d808 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:40.545063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-50f9028b-c4a7-46ed-b994-2819b1b5d808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:40.576429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:40.630810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f11962-5e6d-4ee2-80ee-b964f189c42f None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:40.636454 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 35/67] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:40.637540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9028b-c4a7-46ed-b994-2819b1b5d808 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:40.694739 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a76262-2509-4034-98c4-decdb4019c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:40.696147 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a76262-2509-4034-98c4-decdb4019c1a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:40.696610 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a76262-2509-4034-98c4-decdb4019c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:40.697037 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a76262-2509-4034-98c4-decdb4019c1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:40.710427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a76262-2509-4034-98c4-decdb4019c1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:40.720396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-19a76262-2509-4034-98c4-decdb4019c1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:11:40.720396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:40.720396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:40.720396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:40.720396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:40.720396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:40.720396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:40.720396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:40.720396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:40.720396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:40.720396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:40.720396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:40.720396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:40.720396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:40.720396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:40.720396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:40.721293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:40.721293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:40.721293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:40.721293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:40.721293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:40.721293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:40.721293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:40.721293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:40.721293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:40.721293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:40.721293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:40.721293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:40.721293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:40.721293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:40.721293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:40.721293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:40.722085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:40.722085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:40.722085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:40.722085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:40.722085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:40.722085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:40.722085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:40.722085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:40.722085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:40.722085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:40.722085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:11:40.722085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:11:40.724509 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 33/68] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:40.725636 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a76262-2509-4034-98c4-decdb4019c1a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:40.751195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9028b-c4a7-46ed-b994-2819b1b5d808 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:40.775480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:40.794272 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ba459-af7b-48f0-8ad7-95a0098fe1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:40.795196 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ba459-af7b-48f0-8ad7-95a0098fe1bb None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:40.795491 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ba459-af7b-48f0-8ad7-95a0098fe1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:40.795771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ba459-af7b-48f0-8ad7-95a0098fe1bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:40.807534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ba459-af7b-48f0-8ad7-95a0098fe1bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:40.816486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-b61ba459-af7b-48f0-8ad7-95a0098fe1bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:11:40.816486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:40.816486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:40.816486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:40.816486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:40.816486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:40.816486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:40.816486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:40.816486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:40.816486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:40.816486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:40.816486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:40.816486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:40.816486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:40.816486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:40.816486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:40.817366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:40.817366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:40.817366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:40.817366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:40.817366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:40.817366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:40.817366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:40.817366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:40.817366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:40.817366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:40.817366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:40.817366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:40.817366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:40.817366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:40.817366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:40.817366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:40.818137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:40.818137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:40.818137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:40.818137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:40.818137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:40.818137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:40.818137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:40.818137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:40.818137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:40.818137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:40.818137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:11:40.818137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:11:40.821572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 36/69] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:11:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:40.827214 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ba459-af7b-48f0-8ad7-95a0098fe1bb None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:40.839342 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a76262-2509-4034-98c4-decdb4019c1a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:40.850009 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:40.879030 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704c27f-ddc6-405f-aca5-5a9a033b52ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:40.879851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704c27f-ddc6-405f-aca5-5a9a033b52ca None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:40.880393 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704c27f-ddc6-405f-aca5-5a9a033b52ca None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:40.880675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704c27f-ddc6-405f-aca5-5a9a033b52ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:40.893937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8704c27f-ddc6-405f-aca5-5a9a033b52ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:40.901914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-8704c27f-ddc6-405f-aca5-5a9a033b52ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:11:40.901914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:40.901914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:40.901914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:40.901914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:40.901914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:40.901914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:40.901914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:40.901914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:40.901914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:40.901914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:40.901914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:40.901914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:40.901914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:40.901914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:40.901914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:40.902843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:40.902843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:40.902843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:40.902843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:40.902843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:40.902843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:40.902843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:40.902843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:40.902843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:40.902843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:40.902843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:40.902843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:40.902843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:40.902843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:40.902843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:40.902843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:40.903634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:40.903634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:40.903634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:40.903634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:40.903634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:40.903634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:40.903634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:40.903634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:40.903634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:40.903634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:40.903634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:11:40.903634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:11:40.903634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 34/70] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:11:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:40.904271 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704c27f-ddc6-405f-aca5-5a9a033b52ca None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:40.935294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ba459-af7b-48f0-8ad7-95a0098fe1bb None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:40.971028 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28d84a-d7ee-4c68-bc3b-cd520272f822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:40.972020 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28d84a-d7ee-4c68-bc3b-cd520272f822 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:40.972306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28d84a-d7ee-4c68-bc3b-cd520272f822 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:40.972592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28d84a-d7ee-4c68-bc3b-cd520272f822 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:40.973344 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28d84a-d7ee-4c68-bc3b-cd520272f822 None admin] RBAC: Adding query filter params (name=member) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:40.973656 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28d84a-d7ee-4c68-bc3b-cd520272f822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:40.975187 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28d84a-d7ee-4c68-bc3b-cd520272f822 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:40.984379 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 37/71] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:11:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:40.985801 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582e6ee-daa6-4340-9df1-a8ad1159331b None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:40.985981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582e6ee-daa6-4340-9df1-a8ad1159331b None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:40.986244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582e6ee-daa6-4340-9df1-a8ad1159331b None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:40.987669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 38/72] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:40.988227 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582e6ee-daa6-4340-9df1-a8ad1159331b None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:41.013472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704c27f-ddc6-405f-aca5-5a9a033b52ca None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:41.037267 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:41.048066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d26c71-9e08-4e10-844c-83e23d3fcd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:41.051119 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d26c71-9e08-4e10-844c-83e23d3fcd12 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:41.051119 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d26c71-9e08-4e10-844c-83e23d3fcd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:41.051119 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d26c71-9e08-4e10-844c-83e23d3fcd12 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:41.051119 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d26c71-9e08-4e10-844c-83e23d3fcd12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:41.051119 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d26c71-9e08-4e10-844c-83e23d3fcd12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:41.051850 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d26c71-9e08-4e10-844c-83e23d3fcd12 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:41.060435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 35/73] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:41.061289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d26c71-9e08-4e10-844c-83e23d3fcd12 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:41.095940 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:41.103771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582e6ee-daa6-4340-9df1-a8ad1159331b None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:41.138417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9abbf25-78a5-4452-bb83-d65a2fed843e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:41.139093 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abbf25-78a5-4452-bb83-d65a2fed843e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:41.139378 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abbf25-78a5-4452-bb83-d65a2fed843e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:41.139657 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abbf25-78a5-4452-bb83-d65a2fed843e None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:41.140405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9abbf25-78a5-4452-bb83-d65a2fed843e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:41.140716 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9abbf25-78a5-4452-bb83-d65a2fed843e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:41.142107 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9abbf25-78a5-4452-bb83-d65a2fed843e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:41.149946 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 39/74] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:11:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:41.151090 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fbc1e-b721-40ba-90cd-7ad3d57390bd None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:41.151382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fbc1e-b721-40ba-90cd-7ad3d57390bd None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:41.151664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fbc1e-b721-40ba-90cd-7ad3d57390bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:41.161272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d26c71-9e08-4e10-844c-83e23d3fcd12 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:41.169580 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-200fbc1e-b721-40ba-90cd-7ad3d57390bd None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:41.170911 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-200fbc1e-b721-40ba-90cd-7ad3d57390bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:41.196140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035548de-19eb-4b62-a086-290fccd7ac17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:41.197016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035548de-19eb-4b62-a086-290fccd7ac17 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:41.197302 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035548de-19eb-4b62-a086-290fccd7ac17 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:41.197579 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035548de-19eb-4b62-a086-290fccd7ac17 None admin] PATH_INFO: `/v3/users/83a2209eb05748b8a18c53a472626c39` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:41.212230 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035548de-19eb-4b62-a086-290fccd7ac17 None admin] RBAC: Authorizing `identity:get_user(user_id=83a2209eb05748b8a18c53a472626c39)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:41.213665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035548de-19eb-4b62-a086-290fccd7ac17 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:41.215339 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 36/75] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:41 2022] GET /identity/v3/users/83a2209eb05748b8a18c53a472626c39 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:41.216604 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d134e4-b289-4b37-a459-f681bfd14d4a None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:41.216891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d134e4-b289-4b37-a459-f681bfd14d4a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:41.217166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d134e4-b289-4b37-a459-f681bfd14d4a None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:41.218695 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 37/76] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:41.219480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d134e4-b289-4b37-a459-f681bfd14d4a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:41.254284 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 40/77] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:41.254284 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fbc1e-b721-40ba-90cd-7ad3d57390bd None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:41.276725 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:41.315708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:41.325738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d134e4-b289-4b37-a459-f681bfd14d4a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:41.343754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fbc1e-b721-40ba-90cd-7ad3d57390bd None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:41.360961 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec67865-6bb3-4174-8e87-0c2ea9f2e633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:41.361728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec67865-6bb3-4174-8e87-0c2ea9f2e633 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:41.362014 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec67865-6bb3-4174-8e87-0c2ea9f2e633 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:41.362320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec67865-6bb3-4174-8e87-0c2ea9f2e633 None admin] PATH_INFO: `/v3/users/6049ef02a0334b599d9582b3c2aa2ebf` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:41.363534 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec67865-6bb3-4174-8e87-0c2ea9f2e633 None admin] RBAC: Authorizing `identity:get_user(user_id=6049ef02a0334b599d9582b3c2aa2ebf)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:41.364981 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec67865-6bb3-4174-8e87-0c2ea9f2e633 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:41.367204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 38/78] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:41 2022] GET /identity/v3/users/6049ef02a0334b599d9582b3c2aa2ebf => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:41.367969 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec67865-6bb3-4174-8e87-0c2ea9f2e633 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:41.382838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e75149-7619-4c72-9a22-ee0ec77b0415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:41.383718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e75149-7619-4c72-9a22-ee0ec77b0415 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:41.384003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e75149-7619-4c72-9a22-ee0ec77b0415 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:41.384278 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e75149-7619-4c72-9a22-ee0ec77b0415 None admin] PATH_INFO: `/v3/users/83a2209eb05748b8a18c53a472626c39` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:41.385476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e75149-7619-4c72-9a22-ee0ec77b0415 None admin] RBAC: Authorizing `identity:get_user(user_id=83a2209eb05748b8a18c53a472626c39)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:41.386961 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e75149-7619-4c72-9a22-ee0ec77b0415 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:41.388567 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 41/79] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:41 2022] GET /identity/v3/users/83a2209eb05748b8a18c53a472626c39 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:41.391552 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41ea11-7ba7-403a-9c72-514843dfdd70 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:41.391850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41ea11-7ba7-403a-9c72-514843dfdd70 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:41.392128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41ea11-7ba7-403a-9c72-514843dfdd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:41.408921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-eb41ea11-7ba7-403a-9c72-514843dfdd70 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:41.410052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-eb41ea11-7ba7-403a-9c72-514843dfdd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:41.467600 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec67865-6bb3-4174-8e87-0c2ea9f2e633 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:41.475520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 42/80] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:41.477365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41ea11-7ba7-403a-9c72-514843dfdd70 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:41.504441 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcecffc-571b-41c1-8646-502f4191128c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:41.505359 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcecffc-571b-41c1-8646-502f4191128c None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:41.505670 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcecffc-571b-41c1-8646-502f4191128c None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:41.505957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcecffc-571b-41c1-8646-502f4191128c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:41.507846 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:41.533042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:41.573111 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcecffc-571b-41c1-8646-502f4191128c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:41.574646 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcecffc-571b-41c1-8646-502f4191128c None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:41.575414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41ea11-7ba7-403a-9c72-514843dfdd70 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:41.608317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d79d5a-c8d5-4548-a8b3-dcd21ae4b1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:41.609231 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d79d5a-c8d5-4548-a8b3-dcd21ae4b1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:41.609516 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d79d5a-c8d5-4548-a8b3-dcd21ae4b1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:41.609789 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d79d5a-c8d5-4548-a8b3-dcd21ae4b1d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:41.620172 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d79d5a-c8d5-4548-a8b3-dcd21ae4b1d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:41.627950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-87d79d5a-c8d5-4548-a8b3-dcd21ae4b1d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:11:41.627950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:41.627950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:41.627950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:41.627950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:41.627950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:41.627950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:41.627950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:41.627950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:41.627950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:41.627950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:41.627950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:41.627950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:41.627950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:41.627950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:41.627950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:41.628902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:41.628902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:41.628902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:41.628902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:41.628902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:41.628902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:41.628902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:41.628902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:41.628902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:41.628902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:41.628902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:41.628902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:41.628902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:41.628902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:41.628902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:41.628902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:41.629687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:41.629687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:41.629687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:41.629687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:41.629687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:41.629687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:41.629687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:41.629687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:41.629687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:41.629687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:41.629687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:11:41.629687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:11:41.629687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 43/81] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:41.630663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d79d5a-c8d5-4548-a8b3-dcd21ae4b1d9 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:41.677699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 39/82] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:11:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:41.679713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcecffc-571b-41c1-8646-502f4191128c None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:41.720212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:41.729230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d79d5a-c8d5-4548-a8b3-dcd21ae4b1d9 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:41.736487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:41.763046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1048737-297e-4035-8c23-b58f42dab3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:41.763708 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1048737-297e-4035-8c23-b58f42dab3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:41.764001 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1048737-297e-4035-8c23-b58f42dab3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:41.764284 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1048737-297e-4035-8c23-b58f42dab3f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:41.778191 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcecffc-571b-41c1-8646-502f4191128c None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:41.811758 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d443cb-d383-49eb-92f1-226b8abbf435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:41.812406 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d443cb-d383-49eb-92f1-226b8abbf435 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:41.812693 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d443cb-d383-49eb-92f1-226b8abbf435 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:41.812973 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d443cb-d383-49eb-92f1-226b8abbf435 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:41.830367 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1048737-297e-4035-8c23-b58f42dab3f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:41.831859 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1048737-297e-4035-8c23-b58f42dab3f6 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:41.879782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d443cb-d383-49eb-92f1-226b8abbf435 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:41.882100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d443cb-d383-49eb-92f1-226b8abbf435 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:41.918780 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:41.929672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 44/83] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:11:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:41.930486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1048737-297e-4035-8c23-b58f42dab3f6 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:41.940097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:41.983314 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 40/84] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:11:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:41.984271 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d443cb-d383-49eb-92f1-226b8abbf435 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:42.027390 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1048737-297e-4035-8c23-b58f42dab3f6 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:42.061168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59f50e-558a-478e-b242-67123c149d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:42.061965 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59f50e-558a-478e-b242-67123c149d69 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:42.062250 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59f50e-558a-478e-b242-67123c149d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:42.062554 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59f50e-558a-478e-b242-67123c149d69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:42.063730 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59f50e-558a-478e-b242-67123c149d69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:42.065184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59f50e-558a-478e-b242-67123c149d69 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:42.066850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 45/85] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:11:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:42.067610 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59f50e-558a-478e-b242-67123c149d69 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:42.081736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d443cb-d383-49eb-92f1-226b8abbf435 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:42.118603 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186ceeb-6709-4062-b65f-471c3341856a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:42.119372 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186ceeb-6709-4062-b65f-471c3341856a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:42.119656 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186ceeb-6709-4062-b65f-471c3341856a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:42.119937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186ceeb-6709-4062-b65f-471c3341856a None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:42.120690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186ceeb-6709-4062-b65f-471c3341856a None admin] RBAC: Adding query filter params (name=member) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:42.121003 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186ceeb-6709-4062-b65f-471c3341856a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:42.122437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186ceeb-6709-4062-b65f-471c3341856a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:42.128037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:42.130632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 41/86] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:11:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:42.131486 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186ceeb-6709-4062-b65f-471c3341856a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:42.152350 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:42.170907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59f50e-558a-478e-b242-67123c149d69 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:42.203612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8d6a0-1ce3-446b-b634-6df0409e5cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:42.204419 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8d6a0-1ce3-446b-b634-6df0409e5cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:42.204709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8d6a0-1ce3-446b-b634-6df0409e5cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:42.204993 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8d6a0-1ce3-446b-b634-6df0409e5cc2 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:42.206189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8d6a0-1ce3-446b-b634-6df0409e5cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:42.207686 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8d6a0-1ce3-446b-b634-6df0409e5cc2 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:42.209369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 46/87] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:42 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:42.210166 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8d6a0-1ce3-446b-b634-6df0409e5cc2 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:42.230169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186ceeb-6709-4062-b65f-471c3341856a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:42.263793 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072fc64-c484-413b-a456-5669f05e4300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:42.264663 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072fc64-c484-413b-a456-5669f05e4300 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:42.264950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072fc64-c484-413b-a456-5669f05e4300 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:42.265237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072fc64-c484-413b-a456-5669f05e4300 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:42.266470 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072fc64-c484-413b-a456-5669f05e4300 None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:42.267937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072fc64-c484-413b-a456-5669f05e4300 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:42.269635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 42/88] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:42 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:42.270405 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072fc64-c484-413b-a456-5669f05e4300 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:42.311047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8d6a0-1ce3-446b-b634-6df0409e5cc2 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:42.336599 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:42.345131 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f380a-ddcb-4ef4-88cc-5ebd781e2a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:42.346001 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f380a-ddcb-4ef4-88cc-5ebd781e2a67 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:42.346304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f380a-ddcb-4ef4-88cc-5ebd781e2a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:42.346587 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f380a-ddcb-4ef4-88cc-5ebd781e2a67 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:42.347780 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f380a-ddcb-4ef4-88cc-5ebd781e2a67 None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:42.349241 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f380a-ddcb-4ef4-88cc-5ebd781e2a67 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:42.351699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 47/89] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:42 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:42.352470 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f380a-ddcb-4ef4-88cc-5ebd781e2a67 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:42.370110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:42.382962 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072fc64-c484-413b-a456-5669f05e4300 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:42.417784 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06685dd-75b9-45d9-80ab-d84565210935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:42.418577 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06685dd-75b9-45d9-80ab-d84565210935 None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:42.418931 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06685dd-75b9-45d9-80ab-d84565210935 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:42.419227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06685dd-75b9-45d9-80ab-d84565210935 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:42.419929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06685dd-75b9-45d9-80ab-d84565210935 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:42.421505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06685dd-75b9-45d9-80ab-d84565210935 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:42.451974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f380a-ddcb-4ef4-88cc-5ebd781e2a67 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:42.485349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77510bc5-3657-4197-a211-e11442cd7346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:42.486165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77510bc5-3657-4197-a211-e11442cd7346 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:42.486490 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77510bc5-3657-4197-a211-e11442cd7346 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:42.486774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77510bc5-3657-4197-a211-e11442cd7346 None admin] PATH_INFO: `/v3/users/83a2209eb05748b8a18c53a472626c39` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:42.487993 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77510bc5-3657-4197-a211-e11442cd7346 None admin] RBAC: Authorizing `identity:get_user(user_id=83a2209eb05748b8a18c53a472626c39)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:42.489469 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77510bc5-3657-4197-a211-e11442cd7346 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:42.491162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 48/90] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:42 2022] GET /identity/v3/users/83a2209eb05748b8a18c53a472626c39 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:42.492015 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77510bc5-3657-4197-a211-e11442cd7346 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:42.526297 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 43/91] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:11:42 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:11:42.526297 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06685dd-75b9-45d9-80ab-d84565210935 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:42.561820 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:42.605997 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77510bc5-3657-4197-a211-e11442cd7346 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:42.627035 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06685dd-75b9-45d9-80ab-d84565210935 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:42.644265 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbad17d-f5aa-4fa7-b89e-d1fe3c682c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:42.644943 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbad17d-f5aa-4fa7-b89e-d1fe3c682c72 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:42.645230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbad17d-f5aa-4fa7-b89e-d1fe3c682c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:42.645508 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbad17d-f5aa-4fa7-b89e-d1fe3c682c72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:42.646424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbad17d-f5aa-4fa7-b89e-d1fe3c682c72 None admin] RBAC: Adding query filter params (role.id=d85783606f6649f5b9465f407cafb610, scope.project.id=ba23663aa02d442ab621813797163d82, user.id=83a2209eb05748b8a18c53a472626c39) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:42.646745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbad17d-f5aa-4fa7-b89e-d1fe3c682c72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:42.648833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbad17d-f5aa-4fa7-b89e-d1fe3c682c72 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:42.655002 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 49/92] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:11:42 2022] GET /identity/v3/role_assignments?user.id=83a2209eb05748b8a18c53a472626c39&scope.project.id=ba23663aa02d442ab621813797163d82&role.id=d85783606f6649f5b9465f407cafb610 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:42.655691 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbad17d-f5aa-4fa7-b89e-d1fe3c682c72 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:42.674321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956d091-c12c-41cf-ab26-4ae00a73ea41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:42.674321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956d091-c12c-41cf-ab26-4ae00a73ea41 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:42.674321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956d091-c12c-41cf-ab26-4ae00a73ea41 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:42.674321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956d091-c12c-41cf-ab26-4ae00a73ea41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:42.675350 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956d091-c12c-41cf-ab26-4ae00a73ea41 None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=ba23663aa02d442ab621813797163d82, user.id=6049ef02a0334b599d9582b3c2aa2ebf) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:42.675686 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956d091-c12c-41cf-ab26-4ae00a73ea41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:42.678673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956d091-c12c-41cf-ab26-4ae00a73ea41 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:42.689750 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 44/93] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:11:42 2022] GET /identity/v3/role_assignments?user.id=6049ef02a0334b599d9582b3c2aa2ebf&scope.project.id=ba23663aa02d442ab621813797163d82&role.id=25516be1485c49538544ea8a596e7d17 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:42.689750 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956d091-c12c-41cf-ab26-4ae00a73ea41 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:42.703983 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:42.818153 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbad17d-f5aa-4fa7-b89e-d1fe3c682c72 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:42.840515 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956d091-c12c-41cf-ab26-4ae00a73ea41 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:42.859123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:42.868913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994d8e7-9160-4838-8734-d2d73697ffd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:42.869586 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994d8e7-9160-4838-8734-d2d73697ffd0 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:42.869872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994d8e7-9160-4838-8734-d2d73697ffd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:42.870155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994d8e7-9160-4838-8734-d2d73697ffd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:42.871235 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994d8e7-9160-4838-8734-d2d73697ffd0 None admin] RBAC: Adding query filter params (role.id=2da8d544efe44ce6a41160d5759be402, scope.project.id=ba23663aa02d442ab621813797163d82, user.id=83a2209eb05748b8a18c53a472626c39) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:42.871662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994d8e7-9160-4838-8734-d2d73697ffd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:42.873777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994d8e7-9160-4838-8734-d2d73697ffd0 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:42.883294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 50/94] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:11:42 2022] GET /identity/v3/role_assignments?user.id=83a2209eb05748b8a18c53a472626c39&scope.project.id=ba23663aa02d442ab621813797163d82&role.id=2da8d544efe44ce6a41160d5759be402 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:42.889023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a401f9-1a87-4d82-a0f8-235fca805407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:42.890135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a401f9-1a87-4d82-a0f8-235fca805407 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:42.890674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a401f9-1a87-4d82-a0f8-235fca805407 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:42.890968 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a401f9-1a87-4d82-a0f8-235fca805407 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:42.973376 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a401f9-1a87-4d82-a0f8-235fca805407 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:42.974990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a401f9-1a87-4d82-a0f8-235fca805407 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:42.987052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:43.113448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 45/95] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:11:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:43.119982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994d8e7-9160-4838-8734-d2d73697ffd0 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:43.230098 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994d8e7-9160-4838-8734-d2d73697ffd0 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:43.263753 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a61ca9-b25c-4199-84a3-c595f06fcd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:43.264625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a61ca9-b25c-4199-84a3-c595f06fcd66 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:43.264911 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a61ca9-b25c-4199-84a3-c595f06fcd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:43.265191 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a61ca9-b25c-4199-84a3-c595f06fcd66 None admin] PATH_INFO: `/v3/projects/c018a74ea20d4bbcb71f5c08fcfa6431` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:43.266419 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a61ca9-b25c-4199-84a3-c595f06fcd66 None admin] RBAC: Authorizing `identity:get_project(project_id=c018a74ea20d4bbcb71f5c08fcfa6431)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:43.267949 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a61ca9-b25c-4199-84a3-c595f06fcd66 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:43.269957 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 51/96] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:43 2022] GET /identity/v3/projects/c018a74ea20d4bbcb71f5c08fcfa6431 => generated 312 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:43.276631 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a401f9-1a87-4d82-a0f8-235fca805407 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:43.424148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a401f9-1a87-4d82-a0f8-235fca805407 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:43.433790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:43.460170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4bdff4-42a0-46a6-a802-faf312125a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:43.460844 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bdff4-42a0-46a6-a802-faf312125a1d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:43.461130 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bdff4-42a0-46a6-a802-faf312125a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:43.461409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bdff4-42a0-46a6-a802-faf312125a1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:43.462249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4bdff4-42a0-46a6-a802-faf312125a1d None admin] RBAC: Adding query filter params (role.id=d85783606f6649f5b9465f407cafb610, scope.project.id=c018a74ea20d4bbcb71f5c08fcfa6431, user.id=83a2209eb05748b8a18c53a472626c39) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:43.462590 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4bdff4-42a0-46a6-a802-faf312125a1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:43.464054 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4bdff4-42a0-46a6-a802-faf312125a1d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:43.470129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 46/97] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:11:43 2022] GET /identity/v3/role_assignments?user.id=83a2209eb05748b8a18c53a472626c39&scope.project.id=c018a74ea20d4bbcb71f5c08fcfa6431&role.id=d85783606f6649f5b9465f407cafb610 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:45.558169 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fc8a0-0f7f-4882-9f17-ee26ee6d8769 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:45.559258 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fc8a0-0f7f-4882-9f17-ee26ee6d8769 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:45.559756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fc8a0-0f7f-4882-9f17-ee26ee6d8769 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:45.561590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 52/98] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:45.566627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a615b-59fd-437e-959a-aa763dd6411e None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:45.567453 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a615b-59fd-437e-959a-aa763dd6411e None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:45.567938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a615b-59fd-437e-959a-aa763dd6411e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:45.591313 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-234a615b-59fd-437e-959a-aa763dd6411e None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:45.592881 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-234a615b-59fd-437e-959a-aa763dd6411e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:45.676895 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 47/99] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:45.706992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f1b0a-a867-4bfa-b13d-5b5daa951e79 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:45.707631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f1b0a-a867-4bfa-b13d-5b5daa951e79 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:45.708116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f1b0a-a867-4bfa-b13d-5b5daa951e79 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:45.710411 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 53/100] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:45.711666 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a615b-59fd-437e-959a-aa763dd6411e None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:45.723666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec837fb-f736-4075-b4a1-89519e3d5352 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:45.724663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec837fb-f736-4075-b4a1-89519e3d5352 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:45.725217 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec837fb-f736-4075-b4a1-89519e3d5352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:45.752329 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-8ec837fb-f736-4075-b4a1-89519e3d5352 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:45.753540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8ec837fb-f736-4075-b4a1-89519e3d5352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:45.853906 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:45.856010 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a615b-59fd-437e-959a-aa763dd6411e None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:45.857704 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:45.884571 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 54/101] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:11:45.885991 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b389b81-0de0-40e8-aa39-cda1d2541eff None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:45.886306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b389b81-0de0-40e8-aa39-cda1d2541eff None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:45.886594 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b389b81-0de0-40e8-aa39-cda1d2541eff None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:45.887870 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 55/102] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:45.903882 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c6597-30c3-4e5c-9536-377fc4337ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:45.904697 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c6597-30c3-4e5c-9536-377fc4337ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:45.904988 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c6597-30c3-4e5c-9536-377fc4337ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:45.905273 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c6597-30c3-4e5c-9536-377fc4337ad2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:45.923990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703fb0a-be43-4b5a-9afb-ce1b31fb4592 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:45.924283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703fb0a-be43-4b5a-9afb-ce1b31fb4592 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:45.924570 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703fb0a-be43-4b5a-9afb-ce1b31fb4592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:45.926410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c6597-30c3-4e5c-9536-377fc4337ad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:45.934373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-434c6597-30c3-4e5c-9536-377fc4337ad2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:11:45.934373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:45.934373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:45.934373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:45.934373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:45.934373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:45.934373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:45.934373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:45.934373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:45.934373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:45.934373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:45.934373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:45.934373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:45.934373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:45.934373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:45.934373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:45.935287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:45.935287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:45.935287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:45.935287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:45.935287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:45.935287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:45.935287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:45.935287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:45.935287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:45.935287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:45.935287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:45.935287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:45.935287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:45.935287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:45.935287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:45.935287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:45.936086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:45.936086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:45.936086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:45.936086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:45.936086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:45.936086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:45.936086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:45.936086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:45.936086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:45.936086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:45.936086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:11:45.936086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:11:45.936086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 48/103] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:45.941653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c6597-30c3-4e5c-9536-377fc4337ad2 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:45.975324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-e703fb0a-be43-4b5a-9afb-ce1b31fb4592 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:45.976707 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e703fb0a-be43-4b5a-9afb-ce1b31fb4592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:46.066087 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 56/104] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 20 15:11:46.067007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703fb0a-be43-4b5a-9afb-ce1b31fb4592 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:46.081835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c6597-30c3-4e5c-9536-377fc4337ad2 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:46.134057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aaa186-36af-40e0-87fb-b3b1de1a34f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:46.135435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaa186-36af-40e0-87fb-b3b1de1a34f8 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:46.135993 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaa186-36af-40e0-87fb-b3b1de1a34f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:46.136782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaa186-36af-40e0-87fb-b3b1de1a34f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:46.138446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aaa186-36af-40e0-87fb-b3b1de1a34f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:46.139229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aaa186-36af-40e0-87fb-b3b1de1a34f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:46.141354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aaa186-36af-40e0-87fb-b3b1de1a34f8 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:46.152664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 49/105] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:11:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:46.155106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bbe72-7a98-4439-8c25-5536402da7fd None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:46.155591 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bbe72-7a98-4439-8c25-5536402da7fd None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:46.156027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bbe72-7a98-4439-8c25-5536402da7fd None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:46.157697 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 50/106] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:46.158754 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bbe72-7a98-4439-8c25-5536402da7fd None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:46.189688 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:46.197829 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703fb0a-be43-4b5a-9afb-ce1b31fb4592 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:46.216340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:46.234056 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbe717-66d6-49e8-ba3b-dd42b357ce47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:46.235117 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbe717-66d6-49e8-ba3b-dd42b357ce47 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:46.235481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbe717-66d6-49e8-ba3b-dd42b357ce47 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:46.235741 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbe717-66d6-49e8-ba3b-dd42b357ce47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:46.236974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbe717-66d6-49e8-ba3b-dd42b357ce47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:46.238464 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbe717-66d6-49e8-ba3b-dd42b357ce47 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:46.240304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 57/107] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:11:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:46.245399 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e23a4-fc4c-4484-8d45-26e70e469eab None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:46.245399 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e23a4-fc4c-4484-8d45-26e70e469eab None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:46.245399 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e23a4-fc4c-4484-8d45-26e70e469eab None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:46.245399 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 58/108] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:46.245399 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e23a4-fc4c-4484-8d45-26e70e469eab None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:46.281379 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bbe72-7a98-4439-8c25-5536402da7fd None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:46.319898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e86a48-481f-4f73-aa1c-e0c2445ef174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:46.320749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e86a48-481f-4f73-aa1c-e0c2445ef174 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:46.321008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e86a48-481f-4f73-aa1c-e0c2445ef174 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:46.321294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e86a48-481f-4f73-aa1c-e0c2445ef174 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:46.332158 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e86a48-481f-4f73-aa1c-e0c2445ef174 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:46.339990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-10e86a48-481f-4f73-aa1c-e0c2445ef174 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:11:46.339990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:46.339990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:46.339990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:46.339990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:46.339990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:46.339990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:46.339990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:46.339990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:46.339990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:46.339990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:46.339990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:46.339990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:46.339990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:46.339990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:46.339990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:46.346606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:46.346606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:46.346606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:46.346606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:46.346606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:46.346606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:46.346606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:46.346606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:46.346606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:46.346606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:46.346606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:46.346606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:46.346606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:46.346606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:46.346606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:46.346606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:46.347395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:46.347395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:46.347395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:46.347395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:46.347395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:46.347395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:46.347395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:46.347395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:46.347395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:46.347395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:46.347395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:11:46.347395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:11:46.347395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 51/109] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:11:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:46.347395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe3b23-db39-4027-b7e7-3fe2d032d567 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:46.348102 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe3b23-db39-4027-b7e7-3fe2d032d567 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:46.348102 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe3b23-db39-4027-b7e7-3fe2d032d567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:46.394119 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-bdfe3b23-db39-4027-b7e7-3fe2d032d567 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:46.395277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-bdfe3b23-db39-4027-b7e7-3fe2d032d567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:46.416313 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e23a4-fc4c-4484-8d45-26e70e469eab None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:46.451619 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376f39f-264f-4349-b9b2-2c6905aa8a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:46.452611 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376f39f-264f-4349-b9b2-2c6905aa8a00 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:46.452812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376f39f-264f-4349-b9b2-2c6905aa8a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:46.453061 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376f39f-264f-4349-b9b2-2c6905aa8a00 None admin] PATH_INFO: `/v3/users/83a2209eb05748b8a18c53a472626c39` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:46.454287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376f39f-264f-4349-b9b2-2c6905aa8a00 None admin] RBAC: Authorizing `identity:get_user(user_id=83a2209eb05748b8a18c53a472626c39)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:46.455736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376f39f-264f-4349-b9b2-2c6905aa8a00 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:46.457375 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 59/110] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:46 2022] GET /identity/v3/users/83a2209eb05748b8a18c53a472626c39 => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:46.462626 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376f39f-264f-4349-b9b2-2c6905aa8a00 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:46.467496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 52/111] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:46.468631 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17927d94-b37e-4f47-9fc3-4b3fc0f30b36 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:46.469012 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17927d94-b37e-4f47-9fc3-4b3fc0f30b36 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:46.469233 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17927d94-b37e-4f47-9fc3-4b3fc0f30b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:46.486468 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-17927d94-b37e-4f47-9fc3-4b3fc0f30b36 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:46.487760 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-17927d94-b37e-4f47-9fc3-4b3fc0f30b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:46.498040 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:46.534645 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:46.557892 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 53/112] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:46.564442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17927d94-b37e-4f47-9fc3-4b3fc0f30b36 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:46.582539 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376f39f-264f-4349-b9b2-2c6905aa8a00 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:46.616039 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca262a-b229-47ab-816c-2a9d69a6f2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:46.616705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca262a-b229-47ab-816c-2a9d69a6f2d9 None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:46.617499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca262a-b229-47ab-816c-2a9d69a6f2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:46.617781 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca262a-b229-47ab-816c-2a9d69a6f2d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:46.618555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca262a-b229-47ab-816c-2a9d69a6f2d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:46.620032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca262a-b229-47ab-816c-2a9d69a6f2d9 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:46.664473 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17927d94-b37e-4f47-9fc3-4b3fc0f30b36 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:46.698407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4e29b-5fb0-4a61-b18e-23e1857eaed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:46.699078 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4e29b-5fb0-4a61-b18e-23e1857eaed1 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:46.699365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4e29b-5fb0-4a61-b18e-23e1857eaed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:46.699646 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4e29b-5fb0-4a61-b18e-23e1857eaed1 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:46.700400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4e29b-5fb0-4a61-b18e-23e1857eaed1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:46.700714 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4e29b-5fb0-4a61-b18e-23e1857eaed1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:46.702121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4e29b-5fb0-4a61-b18e-23e1857eaed1 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:46.709985 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 54/113] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:46.710796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4e29b-5fb0-4a61-b18e-23e1857eaed1 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:46.719368 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:46.721349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 60/114] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Jul 20 15:11:46 2022] POST /identity/v3/projects => generated 302 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:11:46.722235 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca262a-b229-47ab-816c-2a9d69a6f2d9 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:46.822355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4e29b-5fb0-4a61-b18e-23e1857eaed1 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:46.833553 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca262a-b229-47ab-816c-2a9d69a6f2d9 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:46.842982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:46.861574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358053a0-87da-4d60-923d-1c057b6acf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:46.862500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358053a0-87da-4d60-923d-1c057b6acf78 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:46.862789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358053a0-87da-4d60-923d-1c057b6acf78 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:46.863086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358053a0-87da-4d60-923d-1c057b6acf78 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:46.864385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358053a0-87da-4d60-923d-1c057b6acf78 None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:46.866059 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358053a0-87da-4d60-923d-1c057b6acf78 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:46.867869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 55/115] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:46 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:46.869228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38cb16-742f-44f2-a2d1-2fccd38eaf57 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:46.869440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38cb16-742f-44f2-a2d1-2fccd38eaf57 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:46.869717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38cb16-742f-44f2-a2d1-2fccd38eaf57 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:46.870998 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 56/116] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:46.871746 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38cb16-742f-44f2-a2d1-2fccd38eaf57 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:46.875815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5028a-d8fe-4128-8fbf-17a6e14b27c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:46.879729 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5028a-d8fe-4128-8fbf-17a6e14b27c3 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:46.879729 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5028a-d8fe-4128-8fbf-17a6e14b27c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:46.879729 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5028a-d8fe-4128-8fbf-17a6e14b27c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:46.891031 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5028a-d8fe-4128-8fbf-17a6e14b27c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:46.900208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-eae5028a-d8fe-4128-8fbf-17a6e14b27c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:11:46.900208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:46.900208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:46.900208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:46.900208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:46.900208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:46.900208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:46.900208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:46.900208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:46.900208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:46.900208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:46.900208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:46.900208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:46.900208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:46.900208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:46.900208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:46.901244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:46.901244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:46.901244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:46.901244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:46.901244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:46.901244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:46.901244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:46.901244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:46.901244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:46.901244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:46.901244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:46.901244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:46.901244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:46.901244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:46.901244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:46.901244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:46.902009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:46.902009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:46.902009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:46.902009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:46.902009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:46.902009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:46.902009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:46.902009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:46.902009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:46.902009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:46.902009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:11:46.902009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:11:46.906410 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 61/117] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:11:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:46.907444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5028a-d8fe-4128-8fbf-17a6e14b27c3 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:46.959069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:46.983781 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38cb16-742f-44f2-a2d1-2fccd38eaf57 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:47.019862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afa1ac-1f72-4274-9d8a-b4f08570f441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:47.020660 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afa1ac-1f72-4274-9d8a-b4f08570f441 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:47.020952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afa1ac-1f72-4274-9d8a-b4f08570f441 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:47.021411 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afa1ac-1f72-4274-9d8a-b4f08570f441 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:47.026957 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5028a-d8fe-4128-8fbf-17a6e14b27c3 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:47.032389 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5afa1ac-1f72-4274-9d8a-b4f08570f441 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:47.041289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-e5afa1ac-1f72-4274-9d8a-b4f08570f441 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:11:47.041289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:47.041289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:47.041289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:47.041289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:47.041289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:47.041289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:47.041289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:47.041289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:47.041289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:47.041289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:47.041289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:47.041289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:47.041289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:47.041289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:47.041289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:47.042145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:47.042145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:47.042145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:47.042145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:47.042145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:47.042145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:47.042145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:47.042145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:47.042145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:47.042145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:47.042145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:47.042145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:47.042145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:47.042145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:47.042145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:47.042145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:47.042967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:47.042967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:47.042967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:47.042967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:47.042967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:47.042967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:47.042967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:47.042967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:47.042967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:47.042967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:47.042967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:11:47.042967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:11:47.042967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 57/118] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:47.043636 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afa1ac-1f72-4274-9d8a-b4f08570f441 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:47.068590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f1d90-0f80-490e-84f7-4522825d7510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:47.070043 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f1d90-0f80-490e-84f7-4522825d7510 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:47.070374 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f1d90-0f80-490e-84f7-4522825d7510 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:47.070661 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f1d90-0f80-490e-84f7-4522825d7510 None admin] PATH_INFO: `/v3/users/6049ef02a0334b599d9582b3c2aa2ebf` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:47.071876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f1d90-0f80-490e-84f7-4522825d7510 None admin] RBAC: Authorizing `identity:get_user(user_id=6049ef02a0334b599d9582b3c2aa2ebf)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:47.073347 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f1d90-0f80-490e-84f7-4522825d7510 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:47.075028 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 62/119] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:46 2022] GET /identity/v3/users/6049ef02a0334b599d9582b3c2aa2ebf => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:47.076173 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b1531-dbd6-4207-8528-c1aecd5f1e93 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:47.076464 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b1531-dbd6-4207-8528-c1aecd5f1e93 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:47.076745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b1531-dbd6-4207-8528-c1aecd5f1e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:47.093198 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-f31b1531-dbd6-4207-8528-c1aecd5f1e93 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:47.094445 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f31b1531-dbd6-4207-8528-c1aecd5f1e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:47.123711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:47.164881 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afa1ac-1f72-4274-9d8a-b4f08570f441 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:47.165810 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 63/120] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:47.166522 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b1531-dbd6-4207-8528-c1aecd5f1e93 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:47.203218 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263aa65-3e60-4916-a06c-2dcb19a8c950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:47.204131 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263aa65-3e60-4916-a06c-2dcb19a8c950 None admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:47.204524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263aa65-3e60-4916-a06c-2dcb19a8c950 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:47.204762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263aa65-3e60-4916-a06c-2dcb19a8c950 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82/users/83a2209eb05748b8a18c53a472626c39/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:47.206720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d263aa65-3e60-4916-a06c-2dcb19a8c950 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba23663aa02d442ab621813797163d82, user_id=83a2209eb05748b8a18c53a472626c39, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:47.208182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d263aa65-3e60-4916-a06c-2dcb19a8c950 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:47.210763 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:47.275165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b1531-dbd6-4207-8528-c1aecd5f1e93 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:47.294289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 58/121] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:11:47 2022] PUT /identity/v3/projects/ba23663aa02d442ab621813797163d82/users/83a2209eb05748b8a18c53a472626c39/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:11:47.294289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263aa65-3e60-4916-a06c-2dcb19a8c950 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:47.326573 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5ec98-8eb6-4d89-971f-e3247e532db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:47.327908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5ec98-8eb6-4d89-971f-e3247e532db6 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:47.328594 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5ec98-8eb6-4d89-971f-e3247e532db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:47.328885 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5ec98-8eb6-4d89-971f-e3247e532db6 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:47.330109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5ec98-8eb6-4d89-971f-e3247e532db6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:47.330960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5ec98-8eb6-4d89-971f-e3247e532db6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:47.332786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5ec98-8eb6-4d89-971f-e3247e532db6 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:47.345409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 64/122] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:11:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:47.349378 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5ec98-8eb6-4d89-971f-e3247e532db6 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:47.384422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:47.425904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263aa65-3e60-4916-a06c-2dcb19a8c950 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:47.466153 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5ec98-8eb6-4d89-971f-e3247e532db6 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:47.466353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa9eb4-1a7d-4895-bff4-5f0c3a9479d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:47.467096 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa9eb4-1a7d-4895-bff4-5f0c3a9479d0 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:47.467384 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa9eb4-1a7d-4895-bff4-5f0c3a9479d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:47.467930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa9eb4-1a7d-4895-bff4-5f0c3a9479d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:47.469738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa9eb4-1a7d-4895-bff4-5f0c3a9479d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:47.470047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa9eb4-1a7d-4895-bff4-5f0c3a9479d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:47.471571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa9eb4-1a7d-4895-bff4-5f0c3a9479d0 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:47.481565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 59/123] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:11:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:47.482414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa9eb4-1a7d-4895-bff4-5f0c3a9479d0 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:47.505563 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6ff0f-459a-43f8-99cf-fecfd8896c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:47.506632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6ff0f-459a-43f8-99cf-fecfd8896c8b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:47.506916 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6ff0f-459a-43f8-99cf-fecfd8896c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:47.507258 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6ff0f-459a-43f8-99cf-fecfd8896c8b None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:47.508508 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6ff0f-459a-43f8-99cf-fecfd8896c8b None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:47.510086 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6ff0f-459a-43f8-99cf-fecfd8896c8b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:47.512464 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 65/124] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:47 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:47.512669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6ff0f-459a-43f8-99cf-fecfd8896c8b None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:47.555707 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:47.611333 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa9eb4-1a7d-4895-bff4-5f0c3a9479d0 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:47.620220 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:47.628989 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6ff0f-459a-43f8-99cf-fecfd8896c8b None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:47.647838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab20b3a-9762-45e5-a25d-e3578e54d328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:47.649001 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab20b3a-9762-45e5-a25d-e3578e54d328 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:47.649569 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab20b3a-9762-45e5-a25d-e3578e54d328 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:47.650068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab20b3a-9762-45e5-a25d-e3578e54d328 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:47.661548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab20b3a-9762-45e5-a25d-e3578e54d328 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:47.670211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-aab20b3a-9762-45e5-a25d-e3578e54d328 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:11:47.670211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:47.670211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:47.670211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:47.670211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:47.670211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:47.670211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:47.670211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:47.670211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:47.670211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:47.670211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:47.670211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:47.670211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:47.670211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:47.670211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:47.670211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:47.679312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:47.679312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:47.679312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:47.679312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:47.679312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:47.679312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:47.679312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:47.679312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:47.679312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:47.679312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:47.679312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:47.679312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:47.679312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:47.679312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:47.679312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:47.679312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:47.680091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:47.680091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:47.680091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:47.680091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:47.680091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:47.680091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:47.680091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:47.680091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:47.680091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:47.680091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:47.680091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:11:47.680091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:11:47.681310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae2b95-31bf-4fb5-a603-8f76b1dc4187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:47.681310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae2b95-31bf-4fb5-a603-8f76b1dc4187 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:47.681310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae2b95-31bf-4fb5-a603-8f76b1dc4187 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:47.681539 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 60/125] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:47.681604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae2b95-31bf-4fb5-a603-8f76b1dc4187 None admin] PATH_INFO: `/v3/users/83a2209eb05748b8a18c53a472626c39` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:47.681672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab20b3a-9762-45e5-a25d-e3578e54d328 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:47.681736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae2b95-31bf-4fb5-a603-8f76b1dc4187 None admin] RBAC: Authorizing `identity:get_user(user_id=83a2209eb05748b8a18c53a472626c39)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:47.681736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae2b95-31bf-4fb5-a603-8f76b1dc4187 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:47.682957 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 66/126] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:47 2022] GET /identity/v3/users/83a2209eb05748b8a18c53a472626c39 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:47.683754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae2b95-31bf-4fb5-a603-8f76b1dc4187 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:47.806487 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae2b95-31bf-4fb5-a603-8f76b1dc4187 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:47.807804 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab20b3a-9762-45e5-a25d-e3578e54d328 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:47.824775 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:47.844143 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf73f7c-6e5d-4871-bb34-b5528779e460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:47.845349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf73f7c-6e5d-4871-bb34-b5528779e460 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:47.845648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebf871-a432-48ee-b097-991671e18c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:47.845966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf73f7c-6e5d-4871-bb34-b5528779e460 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:47.846449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf73f7c-6e5d-4871-bb34-b5528779e460 None admin] PATH_INFO: `/v3/users/eae2a53fd6594125a333aec687f643b5` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:47.846926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebf871-a432-48ee-b097-991671e18c02 None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:47.847297 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebf871-a432-48ee-b097-991671e18c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:47.847670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebf871-a432-48ee-b097-991671e18c02 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82/users/6049ef02a0334b599d9582b3c2aa2ebf/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:47.850999 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebf871-a432-48ee-b097-991671e18c02 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba23663aa02d442ab621813797163d82, user_id=6049ef02a0334b599d9582b3c2aa2ebf, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:47.852071 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebf871-a432-48ee-b097-991671e18c02 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:47.862734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf73f7c-6e5d-4871-bb34-b5528779e460 None admin] RBAC: Authorizing `identity:get_user(user_id=eae2a53fd6594125a333aec687f643b5)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:47.864330 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf73f7c-6e5d-4871-bb34-b5528779e460 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:47.870699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 61/127] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:47 2022] GET /identity/v3/users/eae2a53fd6594125a333aec687f643b5 => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:47.871826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf73f7c-6e5d-4871-bb34-b5528779e460 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:47.946293 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 67/128] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:11:47 2022] PUT /identity/v3/projects/ba23663aa02d442ab621813797163d82/users/6049ef02a0334b599d9582b3c2aa2ebf/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:11:47.946293 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebf871-a432-48ee-b097-991671e18c02 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:47.996926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:48.028233 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf73f7c-6e5d-4871-bb34-b5528779e460 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:48.097584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56627b-19a0-4cf9-84ee-cd782074d5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:48.098201 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56627b-19a0-4cf9-84ee-cd782074d5ec None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:48.098514 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56627b-19a0-4cf9-84ee-cd782074d5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:48.098845 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56627b-19a0-4cf9-84ee-cd782074d5ec None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:48.099628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56627b-19a0-4cf9-84ee-cd782074d5ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:48.099969 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56627b-19a0-4cf9-84ee-cd782074d5ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:48.101371 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56627b-19a0-4cf9-84ee-cd782074d5ec None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:48.116416 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 62/129] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:11:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:48.117300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56627b-19a0-4cf9-84ee-cd782074d5ec None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:48.148378 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebf871-a432-48ee-b097-991671e18c02 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:48.176330 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:48.196339 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8399d5-ff76-44e2-ad9a-34b2ccb6e311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:48.197226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8399d5-ff76-44e2-ad9a-34b2ccb6e311 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:48.197507 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8399d5-ff76-44e2-ad9a-34b2ccb6e311 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:48.197786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8399d5-ff76-44e2-ad9a-34b2ccb6e311 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:48.198999 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8399d5-ff76-44e2-ad9a-34b2ccb6e311 None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:48.200841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8399d5-ff76-44e2-ad9a-34b2ccb6e311 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:48.202496 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 68/130] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:47 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:48.203244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8399d5-ff76-44e2-ad9a-34b2ccb6e311 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:48.259853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56627b-19a0-4cf9-84ee-cd782074d5ec None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:48.296962 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:48.300457 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b2e4d-db3e-4a18-ae15-ef3981be111a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:48.301357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b2e4d-db3e-4a18-ae15-ef3981be111a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:48.301848 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b2e4d-db3e-4a18-ae15-ef3981be111a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:48.302390 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b2e4d-db3e-4a18-ae15-ef3981be111a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:48.306116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8399d5-ff76-44e2-ad9a-34b2ccb6e311 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:48.341509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc648b-a105-47eb-9a33-3a6e629d9563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:48.342420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc648b-a105-47eb-9a33-3a6e629d9563 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:48.342710 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc648b-a105-47eb-9a33-3a6e629d9563 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:48.342991 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc648b-a105-47eb-9a33-3a6e629d9563 None admin] PATH_INFO: `/v3/users/83a2209eb05748b8a18c53a472626c39` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:48.345421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc648b-a105-47eb-9a33-3a6e629d9563 None admin] RBAC: Authorizing `identity:get_user(user_id=83a2209eb05748b8a18c53a472626c39)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:48.345654 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc648b-a105-47eb-9a33-3a6e629d9563 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:48.347676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 69/131] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:48 2022] GET /identity/v3/users/83a2209eb05748b8a18c53a472626c39 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:48.348382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc648b-a105-47eb-9a33-3a6e629d9563 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:48.388638 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b2e4d-db3e-4a18-ae15-ef3981be111a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:48.390557 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b2e4d-db3e-4a18-ae15-ef3981be111a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:48.428941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:48.465810 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc648b-a105-47eb-9a33-3a6e629d9563 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:48.502876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 63/132] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:11:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:48.504108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b2e4d-db3e-4a18-ae15-ef3981be111a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:48.531878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c7658-389a-4eef-a059-83a05699eef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:48.532974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c7658-389a-4eef-a059-83a05699eef0 None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:48.533098 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c7658-389a-4eef-a059-83a05699eef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:48.533340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c7658-389a-4eef-a059-83a05699eef0 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82/users/83a2209eb05748b8a18c53a472626c39/roles/2da8d544efe44ce6a41160d5759be402` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:48.540105 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c7658-389a-4eef-a059-83a05699eef0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba23663aa02d442ab621813797163d82, user_id=83a2209eb05748b8a18c53a472626c39, role_id=2da8d544efe44ce6a41160d5759be402)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:48.541617 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c7658-389a-4eef-a059-83a05699eef0 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:48.619331 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b2e4d-db3e-4a18-ae15-ef3981be111a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:48.620287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 70/133] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:11:48 2022] PUT /identity/v3/projects/ba23663aa02d442ab621813797163d82/users/83a2209eb05748b8a18c53a472626c39/roles/2da8d544efe44ce6a41160d5759be402 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:11:48.621126 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c7658-389a-4eef-a059-83a05699eef0 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:48.662566 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:48.664385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb8fba-de25-4feb-b13a-12ae54614570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:48.665597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb8fba-de25-4feb-b13a-12ae54614570 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:48.666093 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb8fba-de25-4feb-b13a-12ae54614570 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:48.666593 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb8fba-de25-4feb-b13a-12ae54614570 None admin] PATH_INFO: `/v3/projects/c018a74ea20d4bbcb71f5c08fcfa6431` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:48.668419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb8fba-de25-4feb-b13a-12ae54614570 None admin] RBAC: Authorizing `identity:get_project(project_id=c018a74ea20d4bbcb71f5c08fcfa6431)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:48.670347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb8fba-de25-4feb-b13a-12ae54614570 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:48.673463 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 64/134] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:48 2022] GET /identity/v3/projects/c018a74ea20d4bbcb71f5c08fcfa6431 => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:48.680009 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb8fba-de25-4feb-b13a-12ae54614570 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:48.684571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:48.798725 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c7658-389a-4eef-a059-83a05699eef0 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:48.804085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb8fba-de25-4feb-b13a-12ae54614570 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:48.840733 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc07c47-7c9d-46bc-9bad-2e163b6a5998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:48.841602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc07c47-7c9d-46bc-9bad-2e163b6a5998 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:48.841885 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc07c47-7c9d-46bc-9bad-2e163b6a5998 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:48.842159 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc07c47-7c9d-46bc-9bad-2e163b6a5998 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:48.843425 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc07c47-7c9d-46bc-9bad-2e163b6a5998 None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:48.844914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc07c47-7c9d-46bc-9bad-2e163b6a5998 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:48.846009 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda56d06-f90a-47ce-bf42-956685155df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:48.846715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 71/135] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:48 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:48.847417 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda56d06-f90a-47ce-bf42-956685155df9 None admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:48.847915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda56d06-f90a-47ce-bf42-956685155df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:48.848413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda56d06-f90a-47ce-bf42-956685155df9 None admin] PATH_INFO: `/v3/projects/c018a74ea20d4bbcb71f5c08fcfa6431/users/83a2209eb05748b8a18c53a472626c39/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:48.851023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda56d06-f90a-47ce-bf42-956685155df9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c018a74ea20d4bbcb71f5c08fcfa6431, user_id=83a2209eb05748b8a18c53a472626c39, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:48.852714 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc07c47-7c9d-46bc-9bad-2e163b6a5998 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:48.853029 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda56d06-f90a-47ce-bf42-956685155df9 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:48.932485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 65/136] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:11:48 2022] PUT /identity/v3/projects/c018a74ea20d4bbcb71f5c08fcfa6431/users/83a2209eb05748b8a18c53a472626c39/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:11:48.936960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115c8dd-8b9b-404b-bd40-be05f0941cea None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:48.944678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115c8dd-8b9b-404b-bd40-be05f0941cea None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:48.944678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115c8dd-8b9b-404b-bd40-be05f0941cea None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:48.944678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 66/137] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:48.955151 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41665af-f49f-45b2-ac2e-b4cc68626caf None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:48.956019 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41665af-f49f-45b2-ac2e-b4cc68626caf None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:48.956507 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41665af-f49f-45b2-ac2e-b4cc68626caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:48.957930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:48.979876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-e41665af-f49f-45b2-ac2e-b4cc68626caf None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:48.981648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e41665af-f49f-45b2-ac2e-b4cc68626caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:48.995172 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc07c47-7c9d-46bc-9bad-2e163b6a5998 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:49.078821 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:49.120922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 67/138] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:11:49.123054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa5094-ef0e-428d-8a91-9dee0f441b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:49.123745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa5094-ef0e-428d-8a91-9dee0f441b65 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:49.124033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa5094-ef0e-428d-8a91-9dee0f441b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:49.124311 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa5094-ef0e-428d-8a91-9dee0f441b65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:49.125159 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa5094-ef0e-428d-8a91-9dee0f441b65 None admin] RBAC: Adding query filter params (role.id=6c39460198da48b7b86f53b577c698b1, scope.project.id=ba23663aa02d442ab621813797163d82, user.id=eae2a53fd6594125a333aec687f643b5) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:49.125475 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa5094-ef0e-428d-8a91-9dee0f441b65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:49.127067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa5094-ef0e-428d-8a91-9dee0f441b65 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:49.136537 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 72/139] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:11:48 2022] GET /identity/v3/role_assignments?user.id=eae2a53fd6594125a333aec687f643b5&scope.project.id=ba23663aa02d442ab621813797163d82&role.id=6c39460198da48b7b86f53b577c698b1 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:49.166479 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41665af-f49f-45b2-ac2e-b4cc68626caf None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:49.343019 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41665af-f49f-45b2-ac2e-b4cc68626caf None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:49.382835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b4cf6-a58e-4868-820d-cb6e48a0c729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:49.383646 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b4cf6-a58e-4868-820d-cb6e48a0c729 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:49.383937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b4cf6-a58e-4868-820d-cb6e48a0c729 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:49.384223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b4cf6-a58e-4868-820d-cb6e48a0c729 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:49.385497 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b4cf6-a58e-4868-820d-cb6e48a0c729 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:49.386980 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b4cf6-a58e-4868-820d-cb6e48a0c729 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:49.388974 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 68/140] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:11:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:49.394441 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa5094-ef0e-428d-8a91-9dee0f441b65 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:49.507295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa5094-ef0e-428d-8a91-9dee0f441b65 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:49.541316 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82cda1-2210-4b94-8379-bccfbdcb2d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:49.542005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82cda1-2210-4b94-8379-bccfbdcb2d25 None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:49.542314 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82cda1-2210-4b94-8379-bccfbdcb2d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:49.542600 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82cda1-2210-4b94-8379-bccfbdcb2d25 None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:49.543336 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82cda1-2210-4b94-8379-bccfbdcb2d25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:49.544792 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82cda1-2210-4b94-8379-bccfbdcb2d25 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:49.691119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 73/141] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:11:49 2022] POST /identity/v3/users => generated 283 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:11:49.887676 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be441a83-7ded-45b9-8cd3-c4d7816da629 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:49.888203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be441a83-7ded-45b9-8cd3-c4d7816da629 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:49.888278 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be441a83-7ded-45b9-8cd3-c4d7816da629 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:49.889608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 69/142] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:49.895157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eca261-ccc6-482d-8af3-30836ab4e03e None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:49.895448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eca261-ccc6-482d-8af3-30836ab4e03e None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:49.895731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eca261-ccc6-482d-8af3-30836ab4e03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:49.908324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:49.915611 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-01eca261-ccc6-482d-8af3-30836ab4e03e None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:49.917096 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-01eca261-ccc6-482d-8af3-30836ab4e03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:49.993127 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 74/143] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:50.017753 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be441a83-7ded-45b9-8cd3-c4d7816da629 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:50.093856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:50.137189 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be441a83-7ded-45b9-8cd3-c4d7816da629 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:50.175482 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a4ce3-e662-47fc-b47d-fd67eed1111f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:50.176783 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a4ce3-e662-47fc-b47d-fd67eed1111f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:50.177306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a4ce3-e662-47fc-b47d-fd67eed1111f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:50.177787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a4ce3-e662-47fc-b47d-fd67eed1111f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:50.189683 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a4ce3-e662-47fc-b47d-fd67eed1111f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:50.197833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-d59a4ce3-e662-47fc-b47d-fd67eed1111f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:11:50.197833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:50.197833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:50.197833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:50.197833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:50.197833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:50.197833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:50.197833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:50.197833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:50.197833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:50.197833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:50.197833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:50.197833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:50.197833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:50.197833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:50.197833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:50.198843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:50.198843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:50.198843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:50.198843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:50.198843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:50.198843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:50.198843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:50.198843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:50.198843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:50.198843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:50.198843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:50.198843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:50.198843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:50.198843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:50.198843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:50.198843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:50.199790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:50.199790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:50.199790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:50.199790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:50.199790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:50.199790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:50.199790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:50.199790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:50.199790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:50.199790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:50.199790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:11:50.199790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:11:50.201833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 70/144] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:50.207829 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eca261-ccc6-482d-8af3-30836ab4e03e None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:50.324628 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eca261-ccc6-482d-8af3-30836ab4e03e None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:50.361946 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557fc0f-cbf0-4de6-8fae-071901f5ccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:50.362760 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557fc0f-cbf0-4de6-8fae-071901f5ccb3 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:50.363015 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557fc0f-cbf0-4de6-8fae-071901f5ccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:50.363295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557fc0f-cbf0-4de6-8fae-071901f5ccb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:50.364103 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a557fc0f-cbf0-4de6-8fae-071901f5ccb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:50.364418 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a557fc0f-cbf0-4de6-8fae-071901f5ccb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:50.366390 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a557fc0f-cbf0-4de6-8fae-071901f5ccb3 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:50.368739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:50.375367 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 75/145] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:11:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:50.381198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a4ce3-e662-47fc-b47d-fd67eed1111f None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:50.496034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a4ce3-e662-47fc-b47d-fd67eed1111f None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:50.522145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:50.535598 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf7c8b-c975-4827-bd20-863610a9627d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:50.536493 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf7c8b-c975-4827-bd20-863610a9627d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:50.536781 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf7c8b-c975-4827-bd20-863610a9627d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:50.537068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf7c8b-c975-4827-bd20-863610a9627d None admin] PATH_INFO: `/v3/users/83a2209eb05748b8a18c53a472626c39` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:50.538897 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf7c8b-c975-4827-bd20-863610a9627d None admin] RBAC: Authorizing `identity:get_user(user_id=83a2209eb05748b8a18c53a472626c39)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:50.540371 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf7c8b-c975-4827-bd20-863610a9627d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:50.542126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 71/146] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:50 2022] GET /identity/v3/users/83a2209eb05748b8a18c53a472626c39 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:50.547519 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557fc0f-cbf0-4de6-8fae-071901f5ccb3 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:50.657293 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557fc0f-cbf0-4de6-8fae-071901f5ccb3 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:50.692297 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a332f97-addf-4059-b8f4-b62298dea83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:50.693017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a332f97-addf-4059-b8f4-b62298dea83e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:50.693294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a332f97-addf-4059-b8f4-b62298dea83e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:50.693610 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a332f97-addf-4059-b8f4-b62298dea83e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:50.764492 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:50.767818 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a332f97-addf-4059-b8f4-b62298dea83e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:50.769431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a332f97-addf-4059-b8f4-b62298dea83e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:50.875695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 76/147] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:11:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:50.882220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf7c8b-c975-4827-bd20-863610a9627d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:50.995990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf7c8b-c975-4827-bd20-863610a9627d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:51.035337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5998c00-0b2e-479a-9704-e735828f40f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:51.036589 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5998c00-0b2e-479a-9704-e735828f40f7 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:51.037117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5998c00-0b2e-479a-9704-e735828f40f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:51.038117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5998c00-0b2e-479a-9704-e735828f40f7 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:51.039582 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5998c00-0b2e-479a-9704-e735828f40f7 None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:51.041434 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5998c00-0b2e-479a-9704-e735828f40f7 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:51.043402 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 72/148] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:50 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:51.047897 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3d748-dd71-4ce6-b988-d391e1a153f6 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:51.048189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3d748-dd71-4ce6-b988-d391e1a153f6 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:51.048467 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3d748-dd71-4ce6-b988-d391e1a153f6 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:51.049833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 77/149] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:51.054571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5998c00-0b2e-479a-9704-e735828f40f7 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:51.060671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f851bc-511d-4419-97d1-bab109c73535 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:51.060978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f851bc-511d-4419-97d1-bab109c73535 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:51.061235 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f851bc-511d-4419-97d1-bab109c73535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:51.094347 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-e9f851bc-511d-4419-97d1-bab109c73535 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:51.095571 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e9f851bc-511d-4419-97d1-bab109c73535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:51.138675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:51.176997 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5998c00-0b2e-479a-9704-e735828f40f7 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:51.221783 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 78/150] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:11:51.222236 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c9615-709b-4ba6-927a-ebfd043c4ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:51.222925 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c9615-709b-4ba6-927a-ebfd043c4ace None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:51.223210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c9615-709b-4ba6-927a-ebfd043c4ace None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:51.223491 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c9615-709b-4ba6-927a-ebfd043c4ace None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:51.224334 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c9615-709b-4ba6-927a-ebfd043c4ace None admin] RBAC: Adding query filter params (role.id=d85783606f6649f5b9465f407cafb610, scope.project.id=ba23663aa02d442ab621813797163d82, user.id=83a2209eb05748b8a18c53a472626c39) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:51.224647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c9615-709b-4ba6-927a-ebfd043c4ace None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:51.226644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c9615-709b-4ba6-927a-ebfd043c4ace None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:51.239096 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 73/151] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:11:51 2022] GET /identity/v3/role_assignments?user.id=83a2209eb05748b8a18c53a472626c39&scope.project.id=ba23663aa02d442ab621813797163d82&role.id=d85783606f6649f5b9465f407cafb610 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:51.280477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f851bc-511d-4419-97d1-bab109c73535 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:51.314353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:51.401052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f851bc-511d-4419-97d1-bab109c73535 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:51.437538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9784a-2765-4bd5-b491-e8936d05e89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:51.439205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9784a-2765-4bd5-b491-e8936d05e89f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:51.439700 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9784a-2765-4bd5-b491-e8936d05e89f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:51.440226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9784a-2765-4bd5-b491-e8936d05e89f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:51.451979 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9784a-2765-4bd5-b491-e8936d05e89f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:51.461017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-82b9784a-2765-4bd5-b491-e8936d05e89f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:11:51.461017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:51.461017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:51.461017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:51.461017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:51.461017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:51.461017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:51.461017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:51.461017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:51.461017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:51.461017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:51.461017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:51.461017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:51.461017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:51.461017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:51.461017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:51.461966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:51.461966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:51.461966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:51.461966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:51.461966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:51.461966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:51.461966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:51.461966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:51.461966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:51.461966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:51.461966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:51.461966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:51.461966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:51.461966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:51.461966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:51.461966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:51.462786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:51.462786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:51.462786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:51.462786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:51.462786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:51.462786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:51.462786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:51.462786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:51.462786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:51.462786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:51.462786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:11:51.462786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:11:51.465585 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 79/152] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:11:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:51.471058 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c9615-709b-4ba6-927a-ebfd043c4ace None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:51.608130 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c9615-709b-4ba6-927a-ebfd043c4ace None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:51.643035 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:51.660192 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13414341-2c35-4164-9d3f-5c96993ad36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:51.660871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13414341-2c35-4164-9d3f-5c96993ad36d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:51.661161 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13414341-2c35-4164-9d3f-5c96993ad36d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:51.661438 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13414341-2c35-4164-9d3f-5c96993ad36d None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:51.662277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13414341-2c35-4164-9d3f-5c96993ad36d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:51.663069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13414341-2c35-4164-9d3f-5c96993ad36d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:51.664539 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13414341-2c35-4164-9d3f-5c96993ad36d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:51.678627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 74/153] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:51.684204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9784a-2765-4bd5-b491-e8936d05e89f None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:51.813598 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:51.826153 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9784a-2765-4bd5-b491-e8936d05e89f None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:51.861614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54732941-021d-4a67-83ad-e60a3b9c4207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:51.862533 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54732941-021d-4a67-83ad-e60a3b9c4207 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:51.862867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54732941-021d-4a67-83ad-e60a3b9c4207 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:51.863155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54732941-021d-4a67-83ad-e60a3b9c4207 None admin] PATH_INFO: `/v3/users/6049ef02a0334b599d9582b3c2aa2ebf` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:51.864438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54732941-021d-4a67-83ad-e60a3b9c4207 None admin] RBAC: Authorizing `identity:get_user(user_id=6049ef02a0334b599d9582b3c2aa2ebf)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:51.865946 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54732941-021d-4a67-83ad-e60a3b9c4207 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:51.867634 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 80/154] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:51 2022] GET /identity/v3/users/6049ef02a0334b599d9582b3c2aa2ebf => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:51.873368 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13414341-2c35-4164-9d3f-5c96993ad36d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:51.989404 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13414341-2c35-4164-9d3f-5c96993ad36d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:52.026911 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5795d32-1d51-44b9-a909-d92600d46928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:52.028286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5795d32-1d51-44b9-a909-d92600d46928 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:52.028782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5795d32-1d51-44b9-a909-d92600d46928 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:52.029260 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5795d32-1d51-44b9-a909-d92600d46928 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:52.087216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:52.104675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5795d32-1d51-44b9-a909-d92600d46928 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:52.106160 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5795d32-1d51-44b9-a909-d92600d46928 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:52.213785 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 75/155] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:11:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:52.216623 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54732941-021d-4a67-83ad-e60a3b9c4207 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:52.318800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54732941-021d-4a67-83ad-e60a3b9c4207 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:52.355906 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8fb92-308f-4f8c-adc6-ee4359c642bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:52.356834 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8fb92-308f-4f8c-adc6-ee4359c642bf None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:52.357162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8fb92-308f-4f8c-adc6-ee4359c642bf None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:52.357448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8fb92-308f-4f8c-adc6-ee4359c642bf None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:52.358703 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8fb92-308f-4f8c-adc6-ee4359c642bf None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:52.360185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8fb92-308f-4f8c-adc6-ee4359c642bf None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:52.361897 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 81/156] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:52 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:52.368022 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5795d32-1d51-44b9-a909-d92600d46928 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:52.457766 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:52.474587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5795d32-1d51-44b9-a909-d92600d46928 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:52.510581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d200eb-8c61-4e03-9b77-35a27760a5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:52.511247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d200eb-8c61-4e03-9b77-35a27760a5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:52.511535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d200eb-8c61-4e03-9b77-35a27760a5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:52.511819 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d200eb-8c61-4e03-9b77-35a27760a5f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:52.512661 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d200eb-8c61-4e03-9b77-35a27760a5f5 None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=ba23663aa02d442ab621813797163d82, user.id=6049ef02a0334b599d9582b3c2aa2ebf) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:52.512978 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d200eb-8c61-4e03-9b77-35a27760a5f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:52.514591 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d200eb-8c61-4e03-9b77-35a27760a5f5 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:52.521165 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 76/157] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:11:52 2022] GET /identity/v3/role_assignments?user.id=6049ef02a0334b599d9582b3c2aa2ebf&scope.project.id=ba23663aa02d442ab621813797163d82&role.id=25516be1485c49538544ea8a596e7d17 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:52.612030 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8ca12-b20b-4d4b-825b-c6c9c95b31e7 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:52.612343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8ca12-b20b-4d4b-825b-c6c9c95b31e7 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:52.612600 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8ca12-b20b-4d4b-825b-c6c9c95b31e7 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:52.614038 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 82/158] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:52.620011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969b000-977b-4100-ba41-26da3ba210fd None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:52.620326 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969b000-977b-4100-ba41-26da3ba210fd None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:52.620617 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969b000-977b-4100-ba41-26da3ba210fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:52.623739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8a2db-bd0b-4fdc-978e-30a70a960583 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:52.624027 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8a2db-bd0b-4fdc-978e-30a70a960583 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:52.624308 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8a2db-bd0b-4fdc-978e-30a70a960583 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:52.626010 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 83/159] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:52.634485 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5d13-cf06-4576-9ea8-66ad79b0425a None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:52.634800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5d13-cf06-4576-9ea8-66ad79b0425a None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:52.635044 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5d13-cf06-4576-9ea8-66ad79b0425a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:52.661484 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-7dbd5d13-cf06-4576-9ea8-66ad79b0425a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:52.664043 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-7dbd5d13-cf06-4576-9ea8-66ad79b0425a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:52.666101 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-5969b000-977b-4100-ba41-26da3ba210fd None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:52.667347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5969b000-977b-4100-ba41-26da3ba210fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:52.703970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:52.783028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 77/160] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:52.791689 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 84/161] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:52.818309 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969b000-977b-4100-ba41-26da3ba210fd None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:52.822370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd5d13-cf06-4576-9ea8-66ad79b0425a None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:52.934049 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:52.981954 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969b000-977b-4100-ba41-26da3ba210fd None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:53.027382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd5d13-cf06-4576-9ea8-66ad79b0425a None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:53.049326 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf91402-2794-47a4-b4b2-0ac26c763a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:53.050059 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf91402-2794-47a4-b4b2-0ac26c763a9e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:53.050372 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf91402-2794-47a4-b4b2-0ac26c763a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:53.050655 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf91402-2794-47a4-b4b2-0ac26c763a9e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:53.072795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf91402-2794-47a4-b4b2-0ac26c763a9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:53.079955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-ddf91402-2794-47a4-b4b2-0ac26c763a9e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:11:53.079955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:53.079955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:53.079955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:53.079955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:53.079955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:53.079955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:53.079955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:53.079955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:53.079955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:53.079955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:53.079955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:53.079955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:53.079955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:53.079955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:53.079955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:53.081112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:53.081112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:53.081112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:53.081112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:53.081112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:53.081112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:53.081112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:53.081112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:53.081112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:53.081112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:53.081112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:53.081112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:53.081112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:53.081112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:53.081112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:53.081112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:53.082056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:53.082056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:53.082056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:53.082056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:53.082056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:53.082056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:53.082056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:53.082056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:53.082056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:53.082056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:53.082056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:11:53.082056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:11:53.082056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 78/162] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:53.087326 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf91402-2794-47a4-b4b2-0ac26c763a9e None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:53.116046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c322e-9a9c-4e82-8cb9-9e2177eb91e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:53.117104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c322e-9a9c-4e82-8cb9-9e2177eb91e6 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:53.117524 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c322e-9a9c-4e82-8cb9-9e2177eb91e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:53.117934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c322e-9a9c-4e82-8cb9-9e2177eb91e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:53.132820 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c322e-9a9c-4e82-8cb9-9e2177eb91e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:53.141624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-ec3c322e-9a9c-4e82-8cb9-9e2177eb91e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:11:53.141624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:53.141624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:53.141624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:53.141624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:53.141624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:53.141624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:53.141624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:53.141624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:53.141624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:53.141624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:53.141624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:53.141624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:53.141624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:53.141624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:53.141624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:53.142702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:53.142702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:53.142702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:53.142702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:53.142702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:53.142702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:53.142702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:53.142702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:53.142702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:53.142702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:53.142702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:53.142702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:53.142702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:53.142702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:53.142702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:53.142702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:53.143679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:53.143679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:53.143679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:53.143679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:53.143679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:53.143679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:53.143679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:53.143679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:53.143679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:53.143679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:53.143679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:11:53.143679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:11:53.145947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 85/163] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:11:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:53.147658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eae76f-5512-4bfb-bfe4-f81d770fa60e None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:53.148108 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eae76f-5512-4bfb-bfe4-f81d770fa60e None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:53.148538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eae76f-5512-4bfb-bfe4-f81d770fa60e None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:53.150196 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 86/164] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:53.156773 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eae76f-5512-4bfb-bfe4-f81d770fa60e None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:53.181615 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:53.294686 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf91402-2794-47a4-b4b2-0ac26c763a9e None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:53.360237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eae76f-5512-4bfb-bfe4-f81d770fa60e None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:53.370288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a89eca-31c0-4d94-8ec7-8ce48557ee0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:53.370978 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a89eca-31c0-4d94-8ec7-8ce48557ee0c None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:53.371262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a89eca-31c0-4d94-8ec7-8ce48557ee0c None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:53.371575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a89eca-31c0-4d94-8ec7-8ce48557ee0c None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:53.372292 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a89eca-31c0-4d94-8ec7-8ce48557ee0c None admin] RBAC: Adding query filter params (name=member) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:53.372599 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a89eca-31c0-4d94-8ec7-8ce48557ee0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:53.373991 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a89eca-31c0-4d94-8ec7-8ce48557ee0c None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:53.382959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:53.388974 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 79/165] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:11:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:11:53.390233 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e5cc6-4d85-4b42-b711-55bf719b30c4 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:53.390540 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e5cc6-4d85-4b42-b711-55bf719b30c4 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:53.397411 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e5cc6-4d85-4b42-b711-55bf719b30c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:53.409833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-c87e5cc6-4d85-4b42-b711-55bf719b30c4 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:53.411069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-c87e5cc6-4d85-4b42-b711-55bf719b30c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:53.449924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d43c0-e31a-4708-8d4b-460da09a5e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:53.450910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d43c0-e31a-4708-8d4b-460da09a5e0b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:53.450910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d43c0-e31a-4708-8d4b-460da09a5e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:53.451185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d43c0-e31a-4708-8d4b-460da09a5e0b None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:53.452244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d43c0-e31a-4708-8d4b-460da09a5e0b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:53.452244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d43c0-e31a-4708-8d4b-460da09a5e0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:53.453651 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d43c0-e31a-4708-8d4b-460da09a5e0b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:53.463319 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 87/166] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:11:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:53.464136 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d43c0-e31a-4708-8d4b-460da09a5e0b None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:53.507632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 80/167] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:53.508841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e5cc6-4d85-4b42-b711-55bf719b30c4 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:53.603480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:53.609084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e5cc6-4d85-4b42-b711-55bf719b30c4 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:53.634360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d43c0-e31a-4708-8d4b-460da09a5e0b None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:53.642704 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:53.651209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a8d84-cf84-48b5-aaf2-e437a2cd377a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:53.654743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a8d84-cf84-48b5-aaf2-e437a2cd377a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:53.654743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a8d84-cf84-48b5-aaf2-e437a2cd377a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:53.654743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a8d84-cf84-48b5-aaf2-e437a2cd377a None admin] PATH_INFO: `/v3/users/83a2209eb05748b8a18c53a472626c39` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:53.654967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a8d84-cf84-48b5-aaf2-e437a2cd377a None admin] RBAC: Authorizing `identity:get_user(user_id=83a2209eb05748b8a18c53a472626c39)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:53.656363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a8d84-cf84-48b5-aaf2-e437a2cd377a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:53.657976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 81/168] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:53 2022] GET /identity/v3/users/83a2209eb05748b8a18c53a472626c39 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:53.658766 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a8d84-cf84-48b5-aaf2-e437a2cd377a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:53.704745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a933fc7-ab24-4737-8c9d-ec227e52c4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:53.705633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a933fc7-ab24-4737-8c9d-ec227e52c4cf None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:53.705907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a933fc7-ab24-4737-8c9d-ec227e52c4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:53.706177 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a933fc7-ab24-4737-8c9d-ec227e52c4cf None admin] PATH_INFO: `/v3/users/83a2209eb05748b8a18c53a472626c39` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:53.708420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a933fc7-ab24-4737-8c9d-ec227e52c4cf None admin] RBAC: Authorizing `identity:get_user(user_id=83a2209eb05748b8a18c53a472626c39)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:53.709944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a933fc7-ab24-4737-8c9d-ec227e52c4cf None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:53.711630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 88/169] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:53 2022] GET /identity/v3/users/83a2209eb05748b8a18c53a472626c39 => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:53.712649 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a933fc7-ab24-4737-8c9d-ec227e52c4cf None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:53.864694 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a8d84-cf84-48b5-aaf2-e437a2cd377a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:53.879343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a933fc7-ab24-4737-8c9d-ec227e52c4cf None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:53.904541 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eeb07b-0a99-4596-869b-8a2eb25d2005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:53.905740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eeb07b-0a99-4596-869b-8a2eb25d2005 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:53.906226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eeb07b-0a99-4596-869b-8a2eb25d2005 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:53.906727 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eeb07b-0a99-4596-869b-8a2eb25d2005 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:53.916478 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f824f1c-4236-4a4c-b3b2-baafe658dce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:53.917132 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f824f1c-4236-4a4c-b3b2-baafe658dce3 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:53.917411 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f824f1c-4236-4a4c-b3b2-baafe658dce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:53.917685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f824f1c-4236-4a4c-b3b2-baafe658dce3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:53.919628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eeb07b-0a99-4596-869b-8a2eb25d2005 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:53.928528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-c3eeb07b-0a99-4596-869b-8a2eb25d2005 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:11:53.928528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:53.928528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:53.928528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:53.928528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:53.928528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:53.928528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:53.928528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:53.928528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:53.928528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:53.928528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:53.928528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:53.928528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:53.928528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:53.928528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:53.928528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:53.929447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:53.929447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:53.929447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:53.929447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:53.929447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:53.929447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:53.929447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:53.929447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:53.929447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:53.929447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:53.929447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:53.929447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:53.929447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:53.929447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:53.929447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:53.929447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:53.930282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:53.930282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:53.930282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:53.930282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:53.930282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:53.930282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:53.930282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:53.930282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:53.930282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:53.930282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:53.930282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:11:53.930282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:11:53.933040 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 82/170] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:53.933606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:53.935340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eeb07b-0a99-4596-869b-8a2eb25d2005 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:53.992826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f824f1c-4236-4a4c-b3b2-baafe658dce3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:53.994162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f824f1c-4236-4a4c-b3b2-baafe658dce3 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:53.999446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:54.045942 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eeb07b-0a99-4596-869b-8a2eb25d2005 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:54.082991 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1779e-dd82-4681-8807-e5fbb753d7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:54.084531 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1779e-dd82-4681-8807-e5fbb753d7ac None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:54.085043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1779e-dd82-4681-8807-e5fbb753d7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:54.085518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1779e-dd82-4681-8807-e5fbb753d7ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:54.110971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 89/171] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:11:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:54.112139 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057c01c-f099-4278-9a3f-1f9b4ce32747 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:54.112428 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057c01c-f099-4278-9a3f-1f9b4ce32747 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:54.112704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057c01c-f099-4278-9a3f-1f9b4ce32747 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:54.118858 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 90/172] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:54.119652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057c01c-f099-4278-9a3f-1f9b4ce32747 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:54.160525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1779e-dd82-4681-8807-e5fbb753d7ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:54.162873 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1779e-dd82-4681-8807-e5fbb753d7ac None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:54.169386 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:54.231908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:54.234725 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057c01c-f099-4278-9a3f-1f9b4ce32747 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:54.274656 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386aa6b5-5e0a-4d4e-8d35-9b4758acaad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:54.275362 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386aa6b5-5e0a-4d4e-8d35-9b4758acaad9 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:54.275855 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386aa6b5-5e0a-4d4e-8d35-9b4758acaad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:54.276055 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386aa6b5-5e0a-4d4e-8d35-9b4758acaad9 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:54.276805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386aa6b5-5e0a-4d4e-8d35-9b4758acaad9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:54.277191 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386aa6b5-5e0a-4d4e-8d35-9b4758acaad9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:54.278735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386aa6b5-5e0a-4d4e-8d35-9b4758acaad9 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:54.283729 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 83/173] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:11:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:54.288942 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1779e-dd82-4681-8807-e5fbb753d7ac None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:54.289068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 91/174] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:11:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:54.289351 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b753a26-32f3-40d2-bd5b-2a95ba2b7565 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:54.289642 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b753a26-32f3-40d2-bd5b-2a95ba2b7565 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:54.289918 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b753a26-32f3-40d2-bd5b-2a95ba2b7565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:54.307716 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6b753a26-32f3-40d2-bd5b-2a95ba2b7565 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:54.309028 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6b753a26-32f3-40d2-bd5b-2a95ba2b7565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:54.390286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 92/175] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:54.390286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b753a26-32f3-40d2-bd5b-2a95ba2b7565 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:54.404737 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1779e-dd82-4681-8807-e5fbb753d7ac None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:54.418923 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:54.447686 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b309d46-9d4b-4a7f-ba11-3ce340413ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:54.448748 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b309d46-9d4b-4a7f-ba11-3ce340413ead None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:54.449037 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b309d46-9d4b-4a7f-ba11-3ce340413ead None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:54.449315 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b309d46-9d4b-4a7f-ba11-3ce340413ead None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:54.450542 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b309d46-9d4b-4a7f-ba11-3ce340413ead None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:54.452125 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b309d46-9d4b-4a7f-ba11-3ce340413ead None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:54.453985 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 84/176] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:54 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:54.459366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b309d46-9d4b-4a7f-ba11-3ce340413ead None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:54.487460 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:54.489839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b753a26-32f3-40d2-bd5b-2a95ba2b7565 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:54.806213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64d9b9-83f7-499e-b733-abd35eb1ef85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:54.806213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64d9b9-83f7-499e-b733-abd35eb1ef85 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:54.806213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64d9b9-83f7-499e-b733-abd35eb1ef85 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:54.806213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64d9b9-83f7-499e-b733-abd35eb1ef85 None admin] PATH_INFO: `/v3/projects/c018a74ea20d4bbcb71f5c08fcfa6431` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:54.806601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64d9b9-83f7-499e-b733-abd35eb1ef85 None admin] RBAC: Authorizing `identity:get_project(project_id=c018a74ea20d4bbcb71f5c08fcfa6431)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:54.806601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64d9b9-83f7-499e-b733-abd35eb1ef85 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:54.806601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 93/177] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:54 2022] GET /identity/v3/projects/c018a74ea20d4bbcb71f5c08fcfa6431 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:54.806601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca066013-c72d-4d43-b1d8-c3ec9fbacd80 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:54.806601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca066013-c72d-4d43-b1d8-c3ec9fbacd80 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:54.806601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca066013-c72d-4d43-b1d8-c3ec9fbacd80 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:54.806981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 94/178] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:54.806981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca066013-c72d-4d43-b1d8-c3ec9fbacd80 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:54.807112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b309d46-9d4b-4a7f-ba11-3ce340413ead None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:54.807112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe789a-247d-4a82-a702-001f132b531a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:54.807112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe789a-247d-4a82-a702-001f132b531a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:54.807112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe789a-247d-4a82-a702-001f132b531a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:54.807392 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe789a-247d-4a82-a702-001f132b531a None admin] PATH_INFO: `/v3/users/eae2a53fd6594125a333aec687f643b5` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:54.807392 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe789a-247d-4a82-a702-001f132b531a None admin] RBAC: Authorizing `identity:get_user(user_id=eae2a53fd6594125a333aec687f643b5)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:54.807392 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe789a-247d-4a82-a702-001f132b531a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:54.807392 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 85/179] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:54 2022] GET /identity/v3/users/eae2a53fd6594125a333aec687f643b5 => generated 289 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:54.807392 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe789a-247d-4a82-a702-001f132b531a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:54.807705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca066013-c72d-4d43-b1d8-c3ec9fbacd80 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:54.807705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:54.807705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d3663-719f-4be6-a925-88dfe325018f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:54.807705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d3663-719f-4be6-a925-88dfe325018f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:54.807974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d3663-719f-4be6-a925-88dfe325018f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:54.807974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d3663-719f-4be6-a925-88dfe325018f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:54.807974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d3663-719f-4be6-a925-88dfe325018f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:54.807974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-8e7d3663-719f-4be6-a925-88dfe325018f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:11:54.807974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:54.807974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:54.807974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:54.807974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:54.807974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:54.807974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:54.808532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:54.808532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:54.808532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:54.808532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:54.808532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:54.808532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:54.808532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:54.808532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:54.808532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:54.808532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:54.808532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:54.808532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:54.808532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:54.808532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:54.808532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:54.808532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:54.808532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:54.809433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:54.809433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:54.809433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:54.809433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:54.809433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:54.809433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:54.809433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:54.809433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:54.809433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:54.809433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:54.809433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:54.809433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:54.809433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:54.809433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:54.809433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:54.809433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:54.809433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:54.812752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:54.812752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:11:54.812752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:11:54.812752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 95/180] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:11:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:54.812752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c791819-afb9-492e-9bee-8a5dd7a742c1 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:54.812752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c791819-afb9-492e-9bee-8a5dd7a742c1 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:54.812752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c791819-afb9-492e-9bee-8a5dd7a742c1 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:54.813838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:54.813993 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 96/181] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:54.813993 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c791819-afb9-492e-9bee-8a5dd7a742c1 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:54.814300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe789a-247d-4a82-a702-001f132b531a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:54.814300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00c520-073f-47d2-90d3-55631b4ef04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:54.814300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00c520-073f-47d2-90d3-55631b4ef04b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:54.814300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00c520-073f-47d2-90d3-55631b4ef04b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:54.814866 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00c520-073f-47d2-90d3-55631b4ef04b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:54.814866 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00c520-073f-47d2-90d3-55631b4ef04b None admin] RBAC: Adding query filter params (role.id=2da8d544efe44ce6a41160d5759be402, scope.project.id=ba23663aa02d442ab621813797163d82, user.id=83a2209eb05748b8a18c53a472626c39) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:54.814866 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00c520-073f-47d2-90d3-55631b4ef04b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:54.814866 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00c520-073f-47d2-90d3-55631b4ef04b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:54.814866 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 86/182] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:11:54 2022] GET /identity/v3/role_assignments?user.id=83a2209eb05748b8a18c53a472626c39&scope.project.id=ba23663aa02d442ab621813797163d82&role.id=2da8d544efe44ce6a41160d5759be402 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:54.815557 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6cbe2-6163-49b9-bfb9-7eff8a0224c9 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:54.815557 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6cbe2-6163-49b9-bfb9-7eff8a0224c9 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:54.815557 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6cbe2-6163-49b9-bfb9-7eff8a0224c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:54.815557 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-cbf6cbe2-6163-49b9-bfb9-7eff8a0224c9 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:54.815557 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-cbf6cbe2-6163-49b9-bfb9-7eff8a0224c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:54.816243 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c791819-afb9-492e-9bee-8a5dd7a742c1 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:54.842201 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae79750-24ef-4678-b565-634c423e113d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:54.842904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae79750-24ef-4678-b565-634c423e113d None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:54.843184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae79750-24ef-4678-b565-634c423e113d None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:54.843459 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae79750-24ef-4678-b565-634c423e113d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:54.844304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae79750-24ef-4678-b565-634c423e113d None admin] RBAC: Adding query filter params (role.id=d85783606f6649f5b9465f407cafb610, scope.project.id=c018a74ea20d4bbcb71f5c08fcfa6431, user.id=83a2209eb05748b8a18c53a472626c39) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:54.844614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae79750-24ef-4678-b565-634c423e113d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:54.846112 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae79750-24ef-4678-b565-634c423e113d None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:54.861703 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 97/183] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:11:54 2022] GET /identity/v3/role_assignments?user.id=83a2209eb05748b8a18c53a472626c39&scope.project.id=c018a74ea20d4bbcb71f5c08fcfa6431&role.id=d85783606f6649f5b9465f407cafb610 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:54.861703 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae79750-24ef-4678-b565-634c423e113d None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:54.862016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 87/184] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:54.873909 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6cbe2-6163-49b9-bfb9-7eff8a0224c9 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:54.906598 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:55.008787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:55.012914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae79750-24ef-4678-b565-634c423e113d None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:55.022922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6cbe2-6163-49b9-bfb9-7eff8a0224c9 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:55.055744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888f9e1-6cba-40d2-b8ab-c8bce3bb00b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:55.056982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888f9e1-6cba-40d2-b8ab-c8bce3bb00b1 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:55.057418 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888f9e1-6cba-40d2-b8ab-c8bce3bb00b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:55.057694 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888f9e1-6cba-40d2-b8ab-c8bce3bb00b1 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:55.058936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888f9e1-6cba-40d2-b8ab-c8bce3bb00b1 None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:55.060399 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888f9e1-6cba-40d2-b8ab-c8bce3bb00b1 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:55.062098 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 98/185] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:54 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:55.063289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c81ca-79e7-4a0c-9eba-74b8538f0369 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:55.063575 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c81ca-79e7-4a0c-9eba-74b8538f0369 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:55.063800 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfe615-b63a-4a83-8656-7631588c791e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:55.063913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c81ca-79e7-4a0c-9eba-74b8538f0369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:55.064477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfe615-b63a-4a83-8656-7631588c791e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:55.064765 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfe615-b63a-4a83-8656-7631588c791e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:55.065047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfe615-b63a-4a83-8656-7631588c791e None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:55.065804 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adfe615-b63a-4a83-8656-7631588c791e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:55.066113 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adfe615-b63a-4a83-8656-7631588c791e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:55.067542 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adfe615-b63a-4a83-8656-7631588c791e None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:55.077191 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 88/186] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:55.078027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfe615-b63a-4a83-8656-7631588c791e None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:55.081231 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-812c81ca-79e7-4a0c-9eba-74b8538f0369 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:55.082446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-812c81ca-79e7-4a0c-9eba-74b8538f0369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:55.156263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 99/187] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:55.157074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c81ca-79e7-4a0c-9eba-74b8538f0369 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:55.186627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfe615-b63a-4a83-8656-7631588c791e None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:55.189550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:55.220566 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cd045-3d08-4908-a0fd-cba5441edb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:55.221356 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cd045-3d08-4908-a0fd-cba5441edb77 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:55.221640 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cd045-3d08-4908-a0fd-cba5441edb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:55.221917 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cd045-3d08-4908-a0fd-cba5441edb77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:55.223132 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72cd045-3d08-4908-a0fd-cba5441edb77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:55.224583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72cd045-3d08-4908-a0fd-cba5441edb77 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:55.227125 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 89/188] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:11:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:55.227972 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cd045-3d08-4908-a0fd-cba5441edb77 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:55.246434 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:55.260010 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c81ca-79e7-4a0c-9eba-74b8538f0369 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:55.293168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86bd57-cfc6-4bc1-b46c-dd2f73996833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:55.294054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86bd57-cfc6-4bc1-b46c-dd2f73996833 None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:55.294363 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86bd57-cfc6-4bc1-b46c-dd2f73996833 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:55.294636 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86bd57-cfc6-4bc1-b46c-dd2f73996833 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82/users/eae2a53fd6594125a333aec687f643b5/roles/6c39460198da48b7b86f53b577c698b1` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:55.296655 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86bd57-cfc6-4bc1-b46c-dd2f73996833 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba23663aa02d442ab621813797163d82, user_id=eae2a53fd6594125a333aec687f643b5, role_id=6c39460198da48b7b86f53b577c698b1)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:55.298087 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86bd57-cfc6-4bc1-b46c-dd2f73996833 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:55.324049 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cd045-3d08-4908-a0fd-cba5441edb77 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:55.389501 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 100/189] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:11:55 2022] PUT /identity/v3/projects/ba23663aa02d442ab621813797163d82/users/eae2a53fd6594125a333aec687f643b5/roles/6c39460198da48b7b86f53b577c698b1 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:11:55.390336 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86bd57-cfc6-4bc1-b46c-dd2f73996833 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:55.399050 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d5118-38d9-4ae7-98d7-5b26d1ccd704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:55.400606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d5118-38d9-4ae7-98d7-5b26d1ccd704 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:55.400893 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d5118-38d9-4ae7-98d7-5b26d1ccd704 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:55.401170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d5118-38d9-4ae7-98d7-5b26d1ccd704 None admin] PATH_INFO: `/v3/users/ab6e3ab00f4c4e96870913ffd50b64f9` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:55.417975 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d5118-38d9-4ae7-98d7-5b26d1ccd704 None admin] RBAC: Authorizing `identity:get_user(user_id=ab6e3ab00f4c4e96870913ffd50b64f9)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:55.419563 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d5118-38d9-4ae7-98d7-5b26d1ccd704 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:55.421551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 90/190] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:55 2022] GET /identity/v3/users/ab6e3ab00f4c4e96870913ffd50b64f9 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:55.423510 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d5118-38d9-4ae7-98d7-5b26d1ccd704 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:55.491318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:55.508300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86bd57-cfc6-4bc1-b46c-dd2f73996833 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:55.512654 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:55.538195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d5118-38d9-4ae7-98d7-5b26d1ccd704 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:55.551365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a87988-6f99-475b-8a52-88058760582c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:55.552210 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a87988-6f99-475b-8a52-88058760582c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:55.552497 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a87988-6f99-475b-8a52-88058760582c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:55.552810 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a87988-6f99-475b-8a52-88058760582c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:55.564336 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a87988-6f99-475b-8a52-88058760582c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:55.573417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-80a87988-6f99-475b-8a52-88058760582c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:11:55.573417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:55.573417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:55.573417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:55.573417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:55.573417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:55.573417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:55.573417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:55.573417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:55.573417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:55.573417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:55.573417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:55.573417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:55.573417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:55.573417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:55.573417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:55.574463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:55.574463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:55.574463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:55.574463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:55.574463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:55.574463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:55.574463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:55.574463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:55.574463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:55.574463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:55.574463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:55.574463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:55.574463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:55.574463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:55.574463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:55.574463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:55.575295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:55.575295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:55.575295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:55.575295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:55.575295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:55.575295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:55.575295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:55.575295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:55.575295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:55.575295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:55.575295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:11:55.575295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:11:55.576670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 101/191] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:11:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:55.581720 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a87988-6f99-475b-8a52-88058760582c None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:55.583172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90871e5c-2d93-4bea-8cd5-a2516ed0f4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:55.583853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90871e5c-2d93-4bea-8cd5-a2516ed0f4ac None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:55.584694 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90871e5c-2d93-4bea-8cd5-a2516ed0f4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:55.584969 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90871e5c-2d93-4bea-8cd5-a2516ed0f4ac None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:55.585685 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90871e5c-2d93-4bea-8cd5-a2516ed0f4ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:55.587159 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90871e5c-2d93-4bea-8cd5-a2516ed0f4ac None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:55.678295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 91/192] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:11:55 2022] POST /identity/v3/users => generated 297 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:11:55.678295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90871e5c-2d93-4bea-8cd5-a2516ed0f4ac None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:55.690183 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a87988-6f99-475b-8a52-88058760582c None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:55.740798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ef4ca-b9e7-4cd8-84db-a9826fcce740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:55.741479 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ef4ca-b9e7-4cd8-84db-a9826fcce740 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:55.741841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ef4ca-b9e7-4cd8-84db-a9826fcce740 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:55.742358 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ef4ca-b9e7-4cd8-84db-a9826fcce740 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:55.765522 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:55.792150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90871e5c-2d93-4bea-8cd5-a2516ed0f4ac None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:55.825541 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:55.831446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891abff-092e-42b4-811f-5a4b4431059e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:55.832132 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891abff-092e-42b4-811f-5a4b4431059e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:55.832413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891abff-092e-42b4-811f-5a4b4431059e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:55.832690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891abff-092e-42b4-811f-5a4b4431059e None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:55.833533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891abff-092e-42b4-811f-5a4b4431059e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:55.833842 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891abff-092e-42b4-811f-5a4b4431059e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:55.838656 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891abff-092e-42b4-811f-5a4b4431059e None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:55.839268 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ef4ca-b9e7-4cd8-84db-a9826fcce740 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:55.840912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ef4ca-b9e7-4cd8-84db-a9826fcce740 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:55.847349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 92/193] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:11:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:55.856579 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891abff-092e-42b4-811f-5a4b4431059e None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:55.976476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 102/194] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:11:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:55.982798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ef4ca-b9e7-4cd8-84db-a9826fcce740 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:55.987440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891abff-092e-42b4-811f-5a4b4431059e None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:56.031944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b91e4-104b-4f10-bab0-9228ef6d5afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:56.032845 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b91e4-104b-4f10-bab0-9228ef6d5afc None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:56.033130 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b91e4-104b-4f10-bab0-9228ef6d5afc None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:56.033415 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b91e4-104b-4f10-bab0-9228ef6d5afc None admin] PATH_INFO: `/v3/users/ab6e3ab00f4c4e96870913ffd50b64f9` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:56.034700 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b91e4-104b-4f10-bab0-9228ef6d5afc None admin] RBAC: Authorizing `identity:get_user(user_id=ab6e3ab00f4c4e96870913ffd50b64f9)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:56.036184 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b91e4-104b-4f10-bab0-9228ef6d5afc None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:56.039742 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 93/195] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:55 2022] GET /identity/v3/users/ab6e3ab00f4c4e96870913ffd50b64f9 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:56.044147 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b91e4-104b-4f10-bab0-9228ef6d5afc None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:56.050730 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:56.116665 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ef4ca-b9e7-4cd8-84db-a9826fcce740 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:56.136913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:56.161609 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e792f-6956-42ae-8216-cfd2383676bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:56.163507 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e792f-6956-42ae-8216-cfd2383676bf None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:56.164905 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e792f-6956-42ae-8216-cfd2383676bf None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:56.164905 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e792f-6956-42ae-8216-cfd2383676bf None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:56.166060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e792f-6956-42ae-8216-cfd2383676bf None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:56.168015 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e792f-6956-42ae-8216-cfd2383676bf None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:56.170746 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 103/196] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:55 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:56.177035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e792f-6956-42ae-8216-cfd2383676bf None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:56.191682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b91e4-104b-4f10-bab0-9228ef6d5afc None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:56.243986 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702a6c5-cccb-4446-99bd-51d624c67a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:56.244647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702a6c5-cccb-4446-99bd-51d624c67a17 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:56.244928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702a6c5-cccb-4446-99bd-51d624c67a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:56.245208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702a6c5-cccb-4446-99bd-51d624c67a17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:56.286602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e792f-6956-42ae-8216-cfd2383676bf None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:56.301580 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:56.320997 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8702a6c5-cccb-4446-99bd-51d624c67a17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:56.322591 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8702a6c5-cccb-4446-99bd-51d624c67a17 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:56.323405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6e7ce-dbec-47db-aa34-692d090a0986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:56.324283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6e7ce-dbec-47db-aa34-692d090a0986 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:56.324775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6e7ce-dbec-47db-aa34-692d090a0986 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:56.325297 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6e7ce-dbec-47db-aa34-692d090a0986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:56.326349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6e7ce-dbec-47db-aa34-692d090a0986 None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=b271a003871a42fb99fa8b9ca8f9017f, user.id=ab6e3ab00f4c4e96870913ffd50b64f9) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:56.326859 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6e7ce-dbec-47db-aa34-692d090a0986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:56.328490 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6e7ce-dbec-47db-aa34-692d090a0986 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:56.340763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 104/197] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:11:56 2022] GET /identity/v3/role_assignments?user.id=ab6e3ab00f4c4e96870913ffd50b64f9&scope.project.id=b271a003871a42fb99fa8b9ca8f9017f&role.id=25516be1485c49538544ea8a596e7d17 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:11:56.402798 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:56.445552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 94/198] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:11:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 15:11:56.451672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6e7ce-dbec-47db-aa34-692d090a0986 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:56.582155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6e7ce-dbec-47db-aa34-692d090a0986 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:56.630191 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae873e-c56d-416f-869f-84acad58fdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:56.631130 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae873e-c56d-416f-869f-84acad58fdba None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:56.631415 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae873e-c56d-416f-869f-84acad58fdba None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:56.631695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae873e-c56d-416f-869f-84acad58fdba None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:56.639124 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae873e-c56d-416f-869f-84acad58fdba None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:56.640636 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae873e-c56d-416f-869f-84acad58fdba None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:56.642857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 105/199] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:56 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:56.649348 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702a6c5-cccb-4446-99bd-51d624c67a17 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:56.827678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702a6c5-cccb-4446-99bd-51d624c67a17 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:56.873802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53020f81-fdfb-4912-bd88-cff36d74e5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:56.874486 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53020f81-fdfb-4912-bd88-cff36d74e5db None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:56.874773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53020f81-fdfb-4912-bd88-cff36d74e5db None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:56.875051 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53020f81-fdfb-4912-bd88-cff36d74e5db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:56.875882 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53020f81-fdfb-4912-bd88-cff36d74e5db None admin] RBAC: Adding query filter params (role.id=2da8d544efe44ce6a41160d5759be402, scope.project.id=b271a003871a42fb99fa8b9ca8f9017f, user.id=ab6e3ab00f4c4e96870913ffd50b64f9) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:56.876305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53020f81-fdfb-4912-bd88-cff36d74e5db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:56.877716 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53020f81-fdfb-4912-bd88-cff36d74e5db None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:56.884297 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 95/200] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:11:56 2022] GET /identity/v3/role_assignments?user.id=ab6e3ab00f4c4e96870913ffd50b64f9&scope.project.id=b271a003871a42fb99fa8b9ca8f9017f&role.id=2da8d544efe44ce6a41160d5759be402 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:57.828304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb3d07-5c32-492d-8c38-66e230ba3078 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:57.828823 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb3d07-5c32-492d-8c38-66e230ba3078 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:57.828933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb3d07-5c32-492d-8c38-66e230ba3078 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:57.830221 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 106/201] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:57.836116 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df070e17-a869-4cdc-8ed1-595c9fc409c1 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:57.836849 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df070e17-a869-4cdc-8ed1-595c9fc409c1 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:57.837274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df070e17-a869-4cdc-8ed1-595c9fc409c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:57.863220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-df070e17-a869-4cdc-8ed1-595c9fc409c1 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:57.864627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-df070e17-a869-4cdc-8ed1-595c9fc409c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:57.890216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:57.954930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 96/202] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:57.978218 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb3d07-5c32-492d-8c38-66e230ba3078 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:58.038320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:58.122966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb3d07-5c32-492d-8c38-66e230ba3078 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:58.168248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c27348-ac49-40a3-8fa6-0ecdc426f415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:58.169046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27348-ac49-40a3-8fa6-0ecdc426f415 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:58.169332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27348-ac49-40a3-8fa6-0ecdc426f415 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:58.169607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27348-ac49-40a3-8fa6-0ecdc426f415 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:58.187083 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c27348-ac49-40a3-8fa6-0ecdc426f415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:58.195919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-c9c27348-ac49-40a3-8fa6-0ecdc426f415 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:11:58.195919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:58.195919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:58.195919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:58.195919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:58.195919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:58.195919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:58.195919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:58.195919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:58.195919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:58.195919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:58.195919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:58.195919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:58.195919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:58.195919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:58.195919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:58.196842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:58.196842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:58.196842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:58.196842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:58.196842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:58.196842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:58.196842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:58.196842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:58.196842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:58.196842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:58.196842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:58.196842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:58.196842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:58.196842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:58.196842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:58.196842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:58.197675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:58.197675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:58.197675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:58.197675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:58.197675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:58.197675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:58.197675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:58.197675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:58.197675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:58.197675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:58.197675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:11:58.197675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:11:58.197675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 107/203] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:58.203183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df070e17-a869-4cdc-8ed1-595c9fc409c1 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:58.331126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df070e17-a869-4cdc-8ed1-595c9fc409c1 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:58.404877 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d8909-67f2-4eeb-bc14-cb605df948f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:58.405480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d8909-67f2-4eeb-bc14-cb605df948f3 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:58.405764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d8909-67f2-4eeb-bc14-cb605df948f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:58.406043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d8909-67f2-4eeb-bc14-cb605df948f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:58.406830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d8909-67f2-4eeb-bc14-cb605df948f3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:58.407140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d8909-67f2-4eeb-bc14-cb605df948f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:58.408519 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d8909-67f2-4eeb-bc14-cb605df948f3 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:58.421611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 97/204] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:11:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:58.427369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c27348-ac49-40a3-8fa6-0ecdc426f415 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:58.557882 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c27348-ac49-40a3-8fa6-0ecdc426f415 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:58.565012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:58.606971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bedd3d-ee0f-402b-a423-a1ea1df995c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:58.607863 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bedd3d-ee0f-402b-a423-a1ea1df995c6 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:58.608150 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bedd3d-ee0f-402b-a423-a1ea1df995c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:58.608429 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bedd3d-ee0f-402b-a423-a1ea1df995c6 None admin] PATH_INFO: `/v3/users/eae2a53fd6594125a333aec687f643b5` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:58.609679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bedd3d-ee0f-402b-a423-a1ea1df995c6 None admin] RBAC: Authorizing `identity:get_user(user_id=eae2a53fd6594125a333aec687f643b5)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:58.611164 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bedd3d-ee0f-402b-a423-a1ea1df995c6 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:58.612800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 108/205] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:58 2022] GET /identity/v3/users/eae2a53fd6594125a333aec687f643b5 => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:58.618557 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d8909-67f2-4eeb-bc14-cb605df948f3 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:58.640939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:58.662981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc0b6c-3d10-453d-a732-4725dba2d4c1 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:58.663652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc0b6c-3d10-453d-a732-4725dba2d4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:58.664074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc0b6c-3d10-453d-a732-4725dba2d4c1 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:58.665630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 109/206] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:11:58.671453 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b15c27-8afc-48e1-a207-79a1241a4690 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:58.671942 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b15c27-8afc-48e1-a207-79a1241a4690 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:58.672365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b15c27-8afc-48e1-a207-79a1241a4690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:58.698180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-32b15c27-8afc-48e1-a207-79a1241a4690 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:58.699633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-32b15c27-8afc-48e1-a207-79a1241a4690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:58.787559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 110/207] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:11:58.789244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343eaf7-2d7c-449c-a025-92241d4a6961 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:58.789674 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343eaf7-2d7c-449c-a025-92241d4a6961 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:58.790089 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343eaf7-2d7c-449c-a025-92241d4a6961 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:58.791606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 111/208] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:58.793939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d8909-67f2-4eeb-bc14-cb605df948f3 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:58.797215 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff1939-4f43-46dc-8cc8-bfee742f34e1 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:58.797655 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff1939-4f43-46dc-8cc8-bfee742f34e1 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:58.798069 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff1939-4f43-46dc-8cc8-bfee742f34e1 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:58.799802 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 112/209] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:58.801126 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db0d5c-c11d-4c88-98cf-835609b01928 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:58.801608 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db0d5c-c11d-4c88-98cf-835609b01928 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:58.802056 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db0d5c-c11d-4c88-98cf-835609b01928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:58.820915 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-55db0d5c-c11d-4c88-98cf-835609b01928 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:58.822486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-55db0d5c-c11d-4c88-98cf-835609b01928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:58.836620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d1339-8d11-4672-af62-2e6546ea0b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:58.837275 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d1339-8d11-4672-af62-2e6546ea0b24 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:58.837558 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d1339-8d11-4672-af62-2e6546ea0b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:58.837832 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d1339-8d11-4672-af62-2e6546ea0b24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:58.892769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 113/210] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:11:58.900489 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7e59b-0dc5-422f-8aba-bff613517f97 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:58.901236 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7e59b-0dc5-422f-8aba-bff613517f97 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:58.902093 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7e59b-0dc5-422f-8aba-bff613517f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:58.909934 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d1339-8d11-4672-af62-2e6546ea0b24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:58.911990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d1339-8d11-4672-af62-2e6546ea0b24 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:58.924614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-c8c7e59b-0dc5-422f-8aba-bff613517f97 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:58.925727 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c8c7e59b-0dc5-422f-8aba-bff613517f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:58.930905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:58.954925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:58.996030 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 114/211] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:58.996667 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7e59b-0dc5-422f-8aba-bff613517f97 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:59.031773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 98/212] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:11:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:59.032615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d1339-8d11-4672-af62-2e6546ea0b24 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:59.106418 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7e59b-0dc5-422f-8aba-bff613517f97 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:59.141926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4b427-c5b1-414e-8556-082df2ae183b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:59.142759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4b427-c5b1-414e-8556-082df2ae183b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:59.143045 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4b427-c5b1-414e-8556-082df2ae183b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:59.143321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4b427-c5b1-414e-8556-082df2ae183b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:59.144485 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f4b427-c5b1-414e-8556-082df2ae183b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:59.145947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f4b427-c5b1-414e-8556-082df2ae183b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:59.147247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d1339-8d11-4672-af62-2e6546ea0b24 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:59.147638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 115/213] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:11:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:59.148401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4b427-c5b1-414e-8556-082df2ae183b None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:59.170575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:59.183425 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:59.188671 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f199aa-76fb-4816-ab8a-578dba68eaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:59.189469 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f199aa-76fb-4816-ab8a-578dba68eaba None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:59.189749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f199aa-76fb-4816-ab8a-578dba68eaba None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:59.190021 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f199aa-76fb-4816-ab8a-578dba68eaba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:59.200865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f199aa-76fb-4816-ab8a-578dba68eaba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:59.208675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-68f199aa-76fb-4816-ab8a-578dba68eaba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:11:59.208675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:59.208675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:59.208675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:59.208675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:59.208675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:59.208675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:59.208675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:59.208675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:59.208675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:59.208675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:59.208675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:59.208675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:59.208675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:59.208675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:59.208675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:59.209572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:59.209572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:59.209572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:59.209572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:59.209572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:59.209572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:59.209572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:59.209572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:59.209572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:59.209572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:59.209572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:59.209572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:59.209572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:59.209572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:59.209572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:59.209572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:59.215632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:59.215632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:59.215632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:59.215632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:59.215632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:59.215632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:59.215632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:59.215632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:59.215632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:59.215632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:59.215632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:11:59.215632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:11:59.215632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 99/214] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:11:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:59.215632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f199aa-76fb-4816-ab8a-578dba68eaba None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:59.263223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4b427-c5b1-414e-8556-082df2ae183b None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:59.299918 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19947aae-d2d0-4ecc-8dde-dec1fddc3c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:59.300725 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19947aae-d2d0-4ecc-8dde-dec1fddc3c7d None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:59.301015 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19947aae-d2d0-4ecc-8dde-dec1fddc3c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:59.301290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19947aae-d2d0-4ecc-8dde-dec1fddc3c7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:59.312254 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19947aae-d2d0-4ecc-8dde-dec1fddc3c7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:11:59.320224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-19947aae-d2d0-4ecc-8dde-dec1fddc3c7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:11:59.320224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:11:59.320224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:11:59.320224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:11:59.320224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:11:59.320224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:11:59.320224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:11:59.320224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:11:59.320224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:11:59.320224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:11:59.320224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:11:59.320224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:11:59.320224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:11:59.320224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:11:59.320224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:11:59.320224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:11:59.322349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:11:59.322349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:11:59.322349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:11:59.322349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:11:59.322349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:11:59.322349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:11:59.322349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:11:59.322349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:11:59.322349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:11:59.322349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:11:59.322349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:11:59.322349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:11:59.322349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:11:59.322349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:11:59.322349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:11:59.322349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:11:59.323260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:11:59.323260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:11:59.323260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:11:59.323260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:11:59.323260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:11:59.323260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:11:59.323260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:11:59.323260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:11:59.323260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:11:59.323260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:11:59.323260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:11:59.323260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:11:59.324113 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f199aa-76fb-4816-ab8a-578dba68eaba None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:59.324209 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 116/215] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:11:59.324209 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19947aae-d2d0-4ecc-8dde-dec1fddc3c7d None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:59.358005 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f1012-c9f9-49d9-a63a-56fcefcce446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:59.358903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f1012-c9f9-49d9-a63a-56fcefcce446 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:59.359188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f1012-c9f9-49d9-a63a-56fcefcce446 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:59.359467 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f1012-c9f9-49d9-a63a-56fcefcce446 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:59.360666 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f1012-c9f9-49d9-a63a-56fcefcce446 None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:59.362113 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f1012-c9f9-49d9-a63a-56fcefcce446 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:59.363835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 100/216] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:59 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:59.364595 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f1012-c9f9-49d9-a63a-56fcefcce446 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:59.409308 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:59.416478 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:59.424906 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19947aae-d2d0-4ecc-8dde-dec1fddc3c7d None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:59.459085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a09d9-ceec-43ff-8bcf-b60efa1938ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:59.459722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a09d9-ceec-43ff-8bcf-b60efa1938ee None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:59.460012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a09d9-ceec-43ff-8bcf-b60efa1938ee None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:59.460497 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a09d9-ceec-43ff-8bcf-b60efa1938ee None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:59.461415 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a09d9-ceec-43ff-8bcf-b60efa1938ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:59.461515 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f1012-c9f9-49d9-a63a-56fcefcce446 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:59.463261 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a09d9-ceec-43ff-8bcf-b60efa1938ee None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:59.495134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880bf5b-5d83-4bfe-a001-1c3cd04ad652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:59.495796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880bf5b-5d83-4bfe-a001-1c3cd04ad652 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:59.496163 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880bf5b-5d83-4bfe-a001-1c3cd04ad652 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:59.496446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880bf5b-5d83-4bfe-a001-1c3cd04ad652 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:59.497197 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880bf5b-5d83-4bfe-a001-1c3cd04ad652 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:59.497588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880bf5b-5d83-4bfe-a001-1c3cd04ad652 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:59.499000 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880bf5b-5d83-4bfe-a001-1c3cd04ad652 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:59.507218 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 101/217] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:11:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:59.508402 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edeb04-0493-4c8d-a368-6b344635ffbc None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:59.508689 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edeb04-0493-4c8d-a368-6b344635ffbc None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:59.513110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edeb04-0493-4c8d-a368-6b344635ffbc None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:59.513110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 102/218] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:11:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:11:59.513110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edeb04-0493-4c8d-a368-6b344635ffbc None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:59.569231 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 117/219] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:11:59 2022] POST /identity/v3/users => generated 297 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:11:59.570095 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a09d9-ceec-43ff-8bcf-b60efa1938ee None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:59.623079 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edeb04-0493-4c8d-a368-6b344635ffbc None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:59.635491 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:59.663920 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44386bf4-e6e6-4a28-b6c9-409168f3067b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:59.664613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44386bf4-e6e6-4a28-b6c9-409168f3067b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:59.664902 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44386bf4-e6e6-4a28-b6c9-409168f3067b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:59.665189 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44386bf4-e6e6-4a28-b6c9-409168f3067b None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:59.665955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44386bf4-e6e6-4a28-b6c9-409168f3067b None admin] RBAC: Adding query filter params (name=member) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:59.666279 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44386bf4-e6e6-4a28-b6c9-409168f3067b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:59.667716 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44386bf4-e6e6-4a28-b6c9-409168f3067b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:59.677373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 103/220] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:11:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:59.678067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44386bf4-e6e6-4a28-b6c9-409168f3067b None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:59.689720 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a09d9-ceec-43ff-8bcf-b60efa1938ee None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:59.727642 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a22de-1019-48cc-8a0a-e8cbf10685a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:59.728239 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a22de-1019-48cc-8a0a-e8cbf10685a5 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:59.728521 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a22de-1019-48cc-8a0a-e8cbf10685a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:59.728799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a22de-1019-48cc-8a0a-e8cbf10685a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:59.729639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a22de-1019-48cc-8a0a-e8cbf10685a5 None admin] RBAC: Adding query filter params (role.id=6c39460198da48b7b86f53b577c698b1, scope.project.id=ba23663aa02d442ab621813797163d82, user.id=eae2a53fd6594125a333aec687f643b5) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:11:59.729948 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a22de-1019-48cc-8a0a-e8cbf10685a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:59.731574 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a22de-1019-48cc-8a0a-e8cbf10685a5 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:59.737832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 118/221] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:11:59 2022] GET /identity/v3/role_assignments?user.id=eae2a53fd6594125a333aec687f643b5&scope.project.id=ba23663aa02d442ab621813797163d82&role.id=6c39460198da48b7b86f53b577c698b1 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:59.738999 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c24c99-fbf3-4c1b-b176-93fdbb46da2a None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:59.739283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c24c99-fbf3-4c1b-b176-93fdbb46da2a None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:59.739569 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c24c99-fbf3-4c1b-b176-93fdbb46da2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:59.759079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-d8c24c99-fbf3-4c1b-b176-93fdbb46da2a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:11:59.760341 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d8c24c99-fbf3-4c1b-b176-93fdbb46da2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:11:59.774111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:59.814962 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44386bf4-e6e6-4a28-b6c9-409168f3067b None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:11:59.859146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 119/222] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:11:59.860113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c24c99-fbf3-4c1b-b176-93fdbb46da2a None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:59.868232 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb472c-7af1-438f-bba4-ffea4f00b335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:11:59.869135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb472c-7af1-438f-bba4-ffea4f00b335 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:11:59.869460 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb472c-7af1-438f-bba4-ffea4f00b335 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:11:59.869748 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb472c-7af1-438f-bba4-ffea4f00b335 None admin] PATH_INFO: `/v3/users/ab6e3ab00f4c4e96870913ffd50b64f9` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:11:59.871094 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb472c-7af1-438f-bba4-ffea4f00b335 None admin] RBAC: Authorizing `identity:get_user(user_id=ab6e3ab00f4c4e96870913ffd50b64f9)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:11:59.872630 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb472c-7af1-438f-bba4-ffea4f00b335 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:11:59.877420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 104/223] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:59 2022] GET /identity/v3/users/ab6e3ab00f4c4e96870913ffd50b64f9 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:11:59.885652 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb472c-7af1-438f-bba4-ffea4f00b335 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:11:59.942746 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:59.988475 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c24c99-fbf3-4c1b-b176-93fdbb46da2a None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:00.014536 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb472c-7af1-438f-bba4-ffea4f00b335 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:00.034650 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0eba19-1703-4200-ab21-cf44db23e7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:00.035542 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0eba19-1703-4200-ab21-cf44db23e7df None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:00.035828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0eba19-1703-4200-ab21-cf44db23e7df None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:00.036106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0eba19-1703-4200-ab21-cf44db23e7df None admin] PATH_INFO: `/v3/users/8a1fb61c0f724950be189514be7e1253` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:00.054390 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:00.056653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0eba19-1703-4200-ab21-cf44db23e7df None admin] RBAC: Authorizing `identity:get_user(user_id=8a1fb61c0f724950be189514be7e1253)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:00.058138 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0eba19-1703-4200-ab21-cf44db23e7df None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:00.062716 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 120/224] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:11:59 2022] GET /identity/v3/users/8a1fb61c0f724950be189514be7e1253 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:00.062716 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0eba19-1703-4200-ab21-cf44db23e7df None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:00.070185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2aebf-c30b-45db-932a-b94ba8080d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:00.070440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2aebf-c30b-45db-932a-b94ba8080d56 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:00.070730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2aebf-c30b-45db-932a-b94ba8080d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:00.071010 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2aebf-c30b-45db-932a-b94ba8080d56 None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:00.072246 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2aebf-c30b-45db-932a-b94ba8080d56 None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:00.073702 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2aebf-c30b-45db-932a-b94ba8080d56 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:00.076195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 105/225] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:11:59 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:00.076524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2aebf-c30b-45db-932a-b94ba8080d56 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:00.190518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0eba19-1703-4200-ab21-cf44db23e7df None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:00.214534 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2aebf-c30b-45db-932a-b94ba8080d56 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:00.222766 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:00.236646 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de87306-21e3-4300-adf9-5ad541ec89bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:00.238408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de87306-21e3-4300-adf9-5ad541ec89bf None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:00.238408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de87306-21e3-4300-adf9-5ad541ec89bf None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:00.238628 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de87306-21e3-4300-adf9-5ad541ec89bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:00.252595 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81de66-f927-43d8-beb3-641685d192b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:00.252803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de87306-21e3-4300-adf9-5ad541ec89bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:00.253684 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81de66-f927-43d8-beb3-641685d192b6 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:00.254168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81de66-f927-43d8-beb3-641685d192b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:00.254937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81de66-f927-43d8-beb3-641685d192b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:00.262861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-0de87306-21e3-4300-adf9-5ad541ec89bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:00.262861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:00.262861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:00.262861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:00.262861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:00.262861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:00.262861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:00.262861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:00.262861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:00.262861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:00.262861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:00.262861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:00.262861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:00.262861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:00.262861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:00.262861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:00.263772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:00.263772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:00.263772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:00.263772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:00.263772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:00.263772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:00.263772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:00.263772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:00.263772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:00.263772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:00.263772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:00.263772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:00.263772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:00.263772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:00.263772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:00.263772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:00.264631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:00.264631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:00.264631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:00.264631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:00.264631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:00.264631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:00.264631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:00.264631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:00.264631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:00.264631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:00.264631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:00.264631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:00.264631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 121/226] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:12:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:00.265313 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de87306-21e3-4300-adf9-5ad541ec89bf None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:00.341449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf81de66-f927-43d8-beb3-641685d192b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:00.342958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf81de66-f927-43d8-beb3-641685d192b6 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:00.344878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:00.402957 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de87306-21e3-4300-adf9-5ad541ec89bf None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:00.448939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65c4c3-e69c-4359-8e6d-d383cb3da01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:00.450254 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65c4c3-e69c-4359-8e6d-d383cb3da01a None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:00.450667 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65c4c3-e69c-4359-8e6d-d383cb3da01a None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:00.451298 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65c4c3-e69c-4359-8e6d-d383cb3da01a None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f/users/ab6e3ab00f4c4e96870913ffd50b64f9/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:00.454188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65c4c3-e69c-4359-8e6d-d383cb3da01a None admin] RBAC: Authorizing `identity:create_grant(project_id=b271a003871a42fb99fa8b9ca8f9017f, user_id=ab6e3ab00f4c4e96870913ffd50b64f9, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:00.456735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65c4c3-e69c-4359-8e6d-d383cb3da01a None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:00.463947 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 106/227] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:00.465139 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81de66-f927-43d8-beb3-641685d192b6 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:00.474050 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:00.527061 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 122/228] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:12:00 2022] PUT /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f/users/ab6e3ab00f4c4e96870913ffd50b64f9/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:12:00.533469 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65c4c3-e69c-4359-8e6d-d383cb3da01a None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:00.614464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81de66-f927-43d8-beb3-641685d192b6 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:00.680313 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec3a78-7848-4652-a7f9-37f7e34dd4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:00.681008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec3a78-7848-4652-a7f9-37f7e34dd4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:00.681293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec3a78-7848-4652-a7f9-37f7e34dd4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:00.681571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec3a78-7848-4652-a7f9-37f7e34dd4e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:00.682360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec3a78-7848-4652-a7f9-37f7e34dd4e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:00.682675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec3a78-7848-4652-a7f9-37f7e34dd4e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:00.684210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec3a78-7848-4652-a7f9-37f7e34dd4e6 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:00.702004 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 107/229] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:00.709541 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec3a78-7848-4652-a7f9-37f7e34dd4e6 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:00.739827 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65c4c3-e69c-4359-8e6d-d383cb3da01a None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:00.764294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:00.779862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d400e7b-88cb-4015-9996-7b28415e525c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:00.780950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d400e7b-88cb-4015-9996-7b28415e525c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:00.781661 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d400e7b-88cb-4015-9996-7b28415e525c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:00.782085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d400e7b-88cb-4015-9996-7b28415e525c None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:00.783559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d400e7b-88cb-4015-9996-7b28415e525c None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:00.785533 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d400e7b-88cb-4015-9996-7b28415e525c None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:00.788672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 123/230] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:00 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:00.794613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:00.795493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d400e7b-88cb-4015-9996-7b28415e525c None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:00.829109 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec3a78-7848-4652-a7f9-37f7e34dd4e6 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:00.864679 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f322646-b1b4-416f-97a4-94fe987cfa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:00.866334 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f322646-b1b4-416f-97a4-94fe987cfa4a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:00.866728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f322646-b1b4-416f-97a4-94fe987cfa4a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:00.867105 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f322646-b1b4-416f-97a4-94fe987cfa4a None admin] PATH_INFO: `/v3/users/ab6e3ab00f4c4e96870913ffd50b64f9` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:00.868687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f322646-b1b4-416f-97a4-94fe987cfa4a None admin] RBAC: Authorizing `identity:get_user(user_id=ab6e3ab00f4c4e96870913ffd50b64f9)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:00.870410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f322646-b1b4-416f-97a4-94fe987cfa4a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:00.872830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 108/231] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:12:00 2022] GET /identity/v3/users/ab6e3ab00f4c4e96870913ffd50b64f9 => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:00.878291 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f322646-b1b4-416f-97a4-94fe987cfa4a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:00.905538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d400e7b-88cb-4015-9996-7b28415e525c None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:00.948704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2be9a5-ee6b-408f-b87b-d44453bb1a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:00.949615 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2be9a5-ee6b-408f-b87b-d44453bb1a06 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:00.950144 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2be9a5-ee6b-408f-b87b-d44453bb1a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:00.950638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2be9a5-ee6b-408f-b87b-d44453bb1a06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:00.951725 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2be9a5-ee6b-408f-b87b-d44453bb1a06 None admin] RBAC: Adding query filter params (role.id=d85783606f6649f5b9465f407cafb610, scope.project.id=b271a003871a42fb99fa8b9ca8f9017f, user.id=8a1fb61c0f724950be189514be7e1253) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:00.952232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2be9a5-ee6b-408f-b87b-d44453bb1a06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:00.953917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2be9a5-ee6b-408f-b87b-d44453bb1a06 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:00.960359 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 124/232] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:12:00 2022] GET /identity/v3/role_assignments?user.id=8a1fb61c0f724950be189514be7e1253&scope.project.id=b271a003871a42fb99fa8b9ca8f9017f&role.id=d85783606f6649f5b9465f407cafb610 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:12:00.989364 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f322646-b1b4-416f-97a4-94fe987cfa4a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:01.030215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d609a3-1d0c-4eb4-a775-f6e3b40d990e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:01.031138 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d609a3-1d0c-4eb4-a775-f6e3b40d990e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:01.031427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d609a3-1d0c-4eb4-a775-f6e3b40d990e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:01.031705 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d609a3-1d0c-4eb4-a775-f6e3b40d990e None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:01.033121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d609a3-1d0c-4eb4-a775-f6e3b40d990e None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:01.034689 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d609a3-1d0c-4eb4-a775-f6e3b40d990e None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:01.037547 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 109/233] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:00 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:12:01.043550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2be9a5-ee6b-408f-b87b-d44453bb1a06 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:01.122968 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:01.180771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2be9a5-ee6b-408f-b87b-d44453bb1a06 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:01.214396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ea739-b0a2-402a-9750-4d48da41427b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:01.215295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ea739-b0a2-402a-9750-4d48da41427b None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:01.215581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ea739-b0a2-402a-9750-4d48da41427b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:01.215862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ea739-b0a2-402a-9750-4d48da41427b None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f/users/ab6e3ab00f4c4e96870913ffd50b64f9/roles/2da8d544efe44ce6a41160d5759be402` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:01.218113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ea739-b0a2-402a-9750-4d48da41427b None admin] RBAC: Authorizing `identity:create_grant(project_id=b271a003871a42fb99fa8b9ca8f9017f, user_id=ab6e3ab00f4c4e96870913ffd50b64f9, role_id=2da8d544efe44ce6a41160d5759be402)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:01.219606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ea739-b0a2-402a-9750-4d48da41427b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:01.339249 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 125/234] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:12:01 2022] PUT /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f/users/ab6e3ab00f4c4e96870913ffd50b64f9/roles/2da8d544efe44ce6a41160d5759be402 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:12:02.019500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f95b73-9b22-4929-a6c7-9c9cf98c896d None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:02.020031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f95b73-9b22-4929-a6c7-9c9cf98c896d None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:02.020118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f95b73-9b22-4929-a6c7-9c9cf98c896d None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:02.022236 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 110/235] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:02.027412 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2d521-c20d-4f9e-84fd-91bc8e0c7239 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:02.027708 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2d521-c20d-4f9e-84fd-91bc8e0c7239 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:02.027991 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2d521-c20d-4f9e-84fd-91bc8e0c7239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:02.046790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-32a2d521-c20d-4f9e-84fd-91bc8e0c7239 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:02.048009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-32a2d521-c20d-4f9e-84fd-91bc8e0c7239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:02.132933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 126/236] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:02.140958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef85bef-7299-4e10-a5f7-82d769270ec7 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:02.141250 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef85bef-7299-4e10-a5f7-82d769270ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:02.141536 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef85bef-7299-4e10-a5f7-82d769270ec7 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:02.143080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 111/237] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:02.148501 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812e6a9-578a-43f9-b5f3-39350cca8d8a None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:02.148802 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812e6a9-578a-43f9-b5f3-39350cca8d8a None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:02.149086 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812e6a9-578a-43f9-b5f3-39350cca8d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:02.159968 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef85bef-7299-4e10-a5f7-82d769270ec7 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:02.167381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:02.178372 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-c812e6a9-578a-43f9-b5f3-39350cca8d8a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:02.179574 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c812e6a9-578a-43f9-b5f3-39350cca8d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:02.232512 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:02.296943 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 127/238] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:12:02.313868 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef85bef-7299-4e10-a5f7-82d769270ec7 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:02.334451 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812e6a9-578a-43f9-b5f3-39350cca8d8a None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:02.351811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ed099-b09d-41b6-96f1-b4a1e5a75150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:02.352925 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ed099-b09d-41b6-96f1-b4a1e5a75150 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:02.353434 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ed099-b09d-41b6-96f1-b4a1e5a75150 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:02.353902 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ed099-b09d-41b6-96f1-b4a1e5a75150 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:02.355479 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ed099-b09d-41b6-96f1-b4a1e5a75150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:02.357235 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ed099-b09d-41b6-96f1-b4a1e5a75150 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:02.359585 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 112/239] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:02.365135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ed099-b09d-41b6-96f1-b4a1e5a75150 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:02.439316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:02.472269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ed099-b09d-41b6-96f1-b4a1e5a75150 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:02.511869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c75a6-a2b2-44b7-ae28-e3a03611dc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:02.512937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c75a6-a2b2-44b7-ae28-e3a03611dc6b None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:02.513418 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c75a6-a2b2-44b7-ae28-e3a03611dc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:02.513890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c75a6-a2b2-44b7-ae28-e3a03611dc6b None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:02.514956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c75a6-a2b2-44b7-ae28-e3a03611dc6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:02.516717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c75a6-a2b2-44b7-ae28-e3a03611dc6b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:02.539592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812e6a9-578a-43f9-b5f3-39350cca8d8a None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:02.577401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1416aed-651a-4b59-a97a-c6066c2a2cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:02.578558 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1416aed-651a-4b59-a97a-c6066c2a2cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:02.579162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1416aed-651a-4b59-a97a-c6066c2a2cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:02.579672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1416aed-651a-4b59-a97a-c6066c2a2cc3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:02.596055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 113/240] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:12:02 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:12:02.597938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1416aed-651a-4b59-a97a-c6066c2a2cc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:02.612230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-c1416aed-651a-4b59-a97a-c6066c2a2cc3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:12:02.612230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:02.612230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:02.612230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:02.612230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:02.612230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:02.612230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:02.612230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:02.612230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:02.612230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:02.612230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:02.612230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:02.612230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:02.612230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:02.612230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:02.612230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:02.613194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:02.613194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:02.613194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:02.613194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:02.613194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:02.613194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:02.613194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:02.613194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:02.613194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:02.613194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:02.613194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:02.613194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:02.613194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:02.613194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:02.613194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:02.613194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:02.614185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:02.614185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:02.614185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:02.614185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:02.614185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:02.614185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:02.614185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:02.614185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:02.614185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:02.614185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:02.614185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:12:02.614185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:02.622136 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 128/241] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:02.622787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c75a6-a2b2-44b7-ae28-e3a03611dc6b None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:02.769875 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c75a6-a2b2-44b7-ae28-e3a03611dc6b None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:02.799730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:02.814187 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756aa9c8-1c0f-4984-bd18-ff41b4453031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:02.814726 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756aa9c8-1c0f-4984-bd18-ff41b4453031 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:02.815017 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756aa9c8-1c0f-4984-bd18-ff41b4453031 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:02.815295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756aa9c8-1c0f-4984-bd18-ff41b4453031 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:02.816066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756aa9c8-1c0f-4984-bd18-ff41b4453031 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:02.816380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756aa9c8-1c0f-4984-bd18-ff41b4453031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:02.817771 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756aa9c8-1c0f-4984-bd18-ff41b4453031 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:02.827932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 114/242] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:12:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:02.834301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1416aed-651a-4b59-a97a-c6066c2a2cc3 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:02.871640 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:02.945909 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1416aed-651a-4b59-a97a-c6066c2a2cc3 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:02.981648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9baf2-b450-43b2-acd5-f46fb7e994c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:02.983068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9baf2-b450-43b2-acd5-f46fb7e994c7 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:02.983594 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9baf2-b450-43b2-acd5-f46fb7e994c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:02.984068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9baf2-b450-43b2-acd5-f46fb7e994c7 None admin] PATH_INFO: `/v3/users/770a31c527274941b3df3641eafdfe24` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:03.001738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9baf2-b450-43b2-acd5-f46fb7e994c7 None admin] RBAC: Authorizing `identity:get_user(user_id=770a31c527274941b3df3641eafdfe24)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:03.003583 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9baf2-b450-43b2-acd5-f46fb7e994c7 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:03.006702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 129/243] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:12:02 2022] GET /identity/v3/users/770a31c527274941b3df3641eafdfe24 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:03.012134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756aa9c8-1c0f-4984-bd18-ff41b4453031 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:03.086912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656040b-a9bf-4be8-95e2-b0188e9a0f34 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:03.087855 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656040b-a9bf-4be8-95e2-b0188e9a0f34 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:03.088749 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656040b-a9bf-4be8-95e2-b0188e9a0f34 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:03.096687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 130/244] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:03 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:03.103423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465eafe-96e1-437d-8b88-50d0b7fb848b None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:03.104283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465eafe-96e1-437d-8b88-50d0b7fb848b None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:03.105002 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465eafe-96e1-437d-8b88-50d0b7fb848b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:03.128856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-0465eafe-96e1-437d-8b88-50d0b7fb848b None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:03.131294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0465eafe-96e1-437d-8b88-50d0b7fb848b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:03.144776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756aa9c8-1c0f-4984-bd18-ff41b4453031 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:03.187999 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20805f-2313-4c01-af4d-5716b38b3bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:03.188665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20805f-2313-4c01-af4d-5716b38b3bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:03.188950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20805f-2313-4c01-af4d-5716b38b3bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:03.189569 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20805f-2313-4c01-af4d-5716b38b3bf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:03.218125 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 131/245] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:03.254979 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465eafe-96e1-437d-8b88-50d0b7fb848b None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:03.273409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:03.287950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd20805f-2313-4c01-af4d-5716b38b3bf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:03.289543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:03.290129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd20805f-2313-4c01-af4d-5716b38b3bf3 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:03.404200 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465eafe-96e1-437d-8b88-50d0b7fb848b None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:03.426809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 115/246] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:03.435537 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20805f-2313-4c01-af4d-5716b38b3bf3 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:03.467581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a47db3-d642-4c7d-9107-2bc61f057893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:03.468428 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a47db3-d642-4c7d-9107-2bc61f057893 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:03.468709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a47db3-d642-4c7d-9107-2bc61f057893 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:03.468988 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a47db3-d642-4c7d-9107-2bc61f057893 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:03.484475 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a47db3-d642-4c7d-9107-2bc61f057893 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:03.510059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-00a47db3-d642-4c7d-9107-2bc61f057893 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:12:03.510059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:03.510059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:03.510059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:03.510059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:03.510059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:03.510059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:03.510059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:03.510059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:03.510059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:03.510059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:03.510059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:03.510059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:03.510059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:03.510059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:03.510059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:03.511168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:03.511168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:03.511168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:03.511168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:03.511168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:03.511168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:03.511168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:03.511168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:03.511168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:03.511168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:03.511168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:03.511168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:03.511168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:03.511168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:03.511168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:03.511168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:03.511971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:03.511971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:03.511971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:03.511971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:03.511971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:03.511971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:03.511971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:03.511971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:03.511971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:03.511971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:03.511971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:12:03.511971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:03.511971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 132/247] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:12:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:03.522207 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a47db3-d642-4c7d-9107-2bc61f057893 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:03.552301 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:03.563446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20805f-2313-4c01-af4d-5716b38b3bf3 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:03.601763 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e582e2-89d2-4c2c-ad13-bd996c150700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:03.602946 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e582e2-89d2-4c2c-ad13-bd996c150700 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:03.603373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e582e2-89d2-4c2c-ad13-bd996c150700 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:03.603789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e582e2-89d2-4c2c-ad13-bd996c150700 None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:03.605118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e582e2-89d2-4c2c-ad13-bd996c150700 None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:03.606732 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e582e2-89d2-4c2c-ad13-bd996c150700 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:03.608964 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 116/248] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:03 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:03.614790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e582e2-89d2-4c2c-ad13-bd996c150700 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:03.722732 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:03.764500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a47db3-d642-4c7d-9107-2bc61f057893 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:03.817236 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e582e2-89d2-4c2c-ad13-bd996c150700 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:03.832882 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f14e37-b953-4e7e-8b41-69a9ef4abcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:03.833554 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f14e37-b953-4e7e-8b41-69a9ef4abcd5 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:03.835769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f14e37-b953-4e7e-8b41-69a9ef4abcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:03.836045 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f14e37-b953-4e7e-8b41-69a9ef4abcd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:03.836811 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f14e37-b953-4e7e-8b41-69a9ef4abcd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:03.837123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f14e37-b953-4e7e-8b41-69a9ef4abcd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:03.844060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f14e37-b953-4e7e-8b41-69a9ef4abcd5 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:03.865857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 133/249] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:03.867023 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc55621-929c-4db7-87a0-41e9ff9bf265 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:03.867311 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc55621-929c-4db7-87a0-41e9ff9bf265 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:03.867587 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc55621-929c-4db7-87a0-41e9ff9bf265 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:03.880430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 134/250] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:03 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:03.881315 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc55621-929c-4db7-87a0-41e9ff9bf265 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:03.899465 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef091b4b-416e-451c-8d59-03f6fea41f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:03.900131 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef091b4b-416e-451c-8d59-03f6fea41f4b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:03.900413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef091b4b-416e-451c-8d59-03f6fea41f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:03.900689 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef091b4b-416e-451c-8d59-03f6fea41f4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:03.901528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef091b4b-416e-451c-8d59-03f6fea41f4b None admin] RBAC: Adding query filter params (role.id=6c39460198da48b7b86f53b577c698b1, scope.project.id=b271a003871a42fb99fa8b9ca8f9017f, user.id=770a31c527274941b3df3641eafdfe24) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:03.901840 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef091b4b-416e-451c-8d59-03f6fea41f4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:03.903307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef091b4b-416e-451c-8d59-03f6fea41f4b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:03.916489 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 117/251] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:12:03 2022] GET /identity/v3/role_assignments?user.id=770a31c527274941b3df3641eafdfe24&scope.project.id=b271a003871a42fb99fa8b9ca8f9017f&role.id=6c39460198da48b7b86f53b577c698b1 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:12:03.917780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d48b9e-c53c-450e-9b7e-01405806aa74 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:03.918070 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d48b9e-c53c-450e-9b7e-01405806aa74 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:03.925782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d48b9e-c53c-450e-9b7e-01405806aa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:03.968475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-46d48b9e-c53c-450e-9b7e-01405806aa74 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:03.969668 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-46d48b9e-c53c-450e-9b7e-01405806aa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:04.042799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:04.159693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc55621-929c-4db7-87a0-41e9ff9bf265 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:04.163833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 118/252] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:04.204724 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d48b9e-c53c-450e-9b7e-01405806aa74 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:04.239468 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef1ca4-66f5-4b96-b5d2-fac801b0a1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:04.241278 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef1ca4-66f5-4b96-b5d2-fac801b0a1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:04.246830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef1ca4-66f5-4b96-b5d2-fac801b0a1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:04.247461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef1ca4-66f5-4b96-b5d2-fac801b0a1f7 None admin] PATH_INFO: `/v3/users/ab6e3ab00f4c4e96870913ffd50b64f9` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:04.250552 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef1ca4-66f5-4b96-b5d2-fac801b0a1f7 None admin] RBAC: Authorizing `identity:get_user(user_id=ab6e3ab00f4c4e96870913ffd50b64f9)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:04.252296 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef1ca4-66f5-4b96-b5d2-fac801b0a1f7 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:04.259524 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 135/253] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:12:03 2022] GET /identity/v3/users/ab6e3ab00f4c4e96870913ffd50b64f9 => generated 283 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:04.265602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef1ca4-66f5-4b96-b5d2-fac801b0a1f7 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:04.290649 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:04.412726 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d48b9e-c53c-450e-9b7e-01405806aa74 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:04.461146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dd9e3-da36-419b-a0fc-df9a21eb7503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:04.463592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dd9e3-da36-419b-a0fc-df9a21eb7503 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:04.465525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dd9e3-da36-419b-a0fc-df9a21eb7503 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:04.466002 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dd9e3-da36-419b-a0fc-df9a21eb7503 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:04.489449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08dd9e3-da36-419b-a0fc-df9a21eb7503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:04.506121 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef1ca4-66f5-4b96-b5d2-fac801b0a1f7 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:04.507341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-e08dd9e3-da36-419b-a0fc-df9a21eb7503 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:04.507341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:04.507341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:04.507341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:04.507341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:04.507341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:04.507341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:04.507341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:04.507341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:04.507341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:04.507341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:04.507341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:04.507341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:04.507341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:04.507341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:04.507341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:04.508215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:04.508215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:04.508215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:04.508215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:04.508215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:04.508215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:04.508215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:04.508215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:04.508215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:04.508215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:04.508215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:04.508215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:04.508215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:04.508215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:04.508215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:04.508215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:04.509039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:04.509039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:04.509039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:04.509039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:04.509039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:04.509039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:04.509039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:04.509039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:04.509039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:04.509039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:04.509039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:04.509039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:04.514074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 119/254] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:04.515722 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a26a88-d40e-4e94-9686-5eb9c031c713 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:04.521825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a26a88-d40e-4e94-9686-5eb9c031c713 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:04.522386 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a26a88-d40e-4e94-9686-5eb9c031c713 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:04.523931 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 120/255] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:04.524884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a26a88-d40e-4e94-9686-5eb9c031c713 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:04.554645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:04.610122 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2a8bc-47f7-4827-b221-084c0896ae39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:04.612925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2a8bc-47f7-4827-b221-084c0896ae39 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:04.613440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2a8bc-47f7-4827-b221-084c0896ae39 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:04.613910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2a8bc-47f7-4827-b221-084c0896ae39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:04.688559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a26a88-d40e-4e94-9686-5eb9c031c713 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:04.728893 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2a8bc-47f7-4827-b221-084c0896ae39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:04.730778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2a8bc-47f7-4827-b221-084c0896ae39 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:04.746854 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:04.789940 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37689a13-9715-4249-903e-a4ef40616288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:04.791047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37689a13-9715-4249-903e-a4ef40616288 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:04.792038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37689a13-9715-4249-903e-a4ef40616288 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:04.796087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37689a13-9715-4249-903e-a4ef40616288 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:04.797065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37689a13-9715-4249-903e-a4ef40616288 None admin] RBAC: Adding query filter params (name=member) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:04.797567 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37689a13-9715-4249-903e-a4ef40616288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:04.799221 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37689a13-9715-4249-903e-a4ef40616288 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:04.820479 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 121/256] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:12:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:04.821713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4457b-597f-4d57-bb89-527abbe75cd6 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:04.822001 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4457b-597f-4d57-bb89-527abbe75cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:04.822298 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4457b-597f-4d57-bb89-527abbe75cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:04.869435 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 136/257] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:04.870663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2a8bc-47f7-4827-b221-084c0896ae39 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:04.871330 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-98a4457b-597f-4d57-bb89-527abbe75cd6 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:04.872590 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-98a4457b-597f-4d57-bb89-527abbe75cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:04.976062 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 122/258] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:04.976838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4457b-597f-4d57-bb89-527abbe75cd6 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:04.986503 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:05.078349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2a8bc-47f7-4827-b221-084c0896ae39 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:05.094945 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:05.139530 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d891c-dfa5-4385-b4b4-605e72c132fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:05.140425 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d891c-dfa5-4385-b4b4-605e72c132fa None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:05.140710 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d891c-dfa5-4385-b4b4-605e72c132fa None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:05.140989 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d891c-dfa5-4385-b4b4-605e72c132fa None admin] PATH_INFO: `/v3/users/8a1fb61c0f724950be189514be7e1253` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:05.147604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d891c-dfa5-4385-b4b4-605e72c132fa None admin] RBAC: Authorizing `identity:get_user(user_id=8a1fb61c0f724950be189514be7e1253)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:05.149375 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d891c-dfa5-4385-b4b4-605e72c132fa None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:05.151322 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 137/259] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:12:04 2022] GET /identity/v3/users/8a1fb61c0f724950be189514be7e1253 => generated 297 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:05.152279 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d891c-dfa5-4385-b4b4-605e72c132fa None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:05.170511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4457b-597f-4d57-bb89-527abbe75cd6 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:05.214496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e252a-cb8a-481e-8cb0-ac1cc47799a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:05.215372 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e252a-cb8a-481e-8cb0-ac1cc47799a2 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:05.215653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e252a-cb8a-481e-8cb0-ac1cc47799a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:05.215927 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e252a-cb8a-481e-8cb0-ac1cc47799a2 None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:05.220199 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e252a-cb8a-481e-8cb0-ac1cc47799a2 None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:05.221672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e252a-cb8a-481e-8cb0-ac1cc47799a2 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:05.231490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 123/260] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:04 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:05.232305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e252a-cb8a-481e-8cb0-ac1cc47799a2 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:05.343179 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d891c-dfa5-4385-b4b4-605e72c132fa None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:05.387983 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51e7ef-9217-411f-a964-7bb39ef1f4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:05.388812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51e7ef-9217-411f-a964-7bb39ef1f4fa None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:05.389093 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51e7ef-9217-411f-a964-7bb39ef1f4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:05.389369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51e7ef-9217-411f-a964-7bb39ef1f4fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:05.404162 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:05.418727 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51e7ef-9217-411f-a964-7bb39ef1f4fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:05.439357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-ad51e7ef-9217-411f-a964-7bb39ef1f4fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:05.439357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:05.439357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:05.439357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:05.439357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:05.439357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:05.439357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:05.439357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:05.439357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:05.439357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:05.439357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:05.439357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:05.439357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:05.439357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:05.439357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:05.439357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:05.440300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:05.440300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:05.440300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:05.440300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:05.440300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:05.440300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:05.440300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:05.440300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:05.440300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:05.440300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:05.440300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:05.440300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:05.440300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:05.440300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:05.440300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:05.440300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:05.441133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:05.441133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:05.441133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:05.441133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:05.441133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:05.441133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:05.441133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:05.441133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:05.441133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:05.441133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:05.441133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:05.441133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:05.445842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 138/261] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:12:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:05.445842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51e7ef-9217-411f-a964-7bb39ef1f4fa None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:05.471398 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e252a-cb8a-481e-8cb0-ac1cc47799a2 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:05.489258 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:05.529811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9768cd-4f07-4381-9c1f-80c38c38ca4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:05.531070 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9768cd-4f07-4381-9c1f-80c38c38ca4f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:05.531556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9768cd-4f07-4381-9c1f-80c38c38ca4f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:05.532025 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9768cd-4f07-4381-9c1f-80c38c38ca4f None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:05.533476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9768cd-4f07-4381-9c1f-80c38c38ca4f None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:05.539640 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9768cd-4f07-4381-9c1f-80c38c38ca4f None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:05.541936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 124/262] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:05 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:05.542996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9768cd-4f07-4381-9c1f-80c38c38ca4f None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:05.554379 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51e7ef-9217-411f-a964-7bb39ef1f4fa None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:05.597709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d395d-dcba-4e3e-8a30-6cffebae4a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:05.598422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d395d-dcba-4e3e-8a30-6cffebae4a33 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:05.598713 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d395d-dcba-4e3e-8a30-6cffebae4a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:05.598992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d395d-dcba-4e3e-8a30-6cffebae4a33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:05.599865 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d395d-dcba-4e3e-8a30-6cffebae4a33 None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=b271a003871a42fb99fa8b9ca8f9017f, user.id=ab6e3ab00f4c4e96870913ffd50b64f9) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:05.600180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d395d-dcba-4e3e-8a30-6cffebae4a33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:05.601652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d395d-dcba-4e3e-8a30-6cffebae4a33 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:05.615783 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 139/263] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:12:05 2022] GET /identity/v3/role_assignments?user.id=ab6e3ab00f4c4e96870913ffd50b64f9&scope.project.id=b271a003871a42fb99fa8b9ca8f9017f&role.id=25516be1485c49538544ea8a596e7d17 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:05.616611 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d395d-dcba-4e3e-8a30-6cffebae4a33 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:05.706696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9768cd-4f07-4381-9c1f-80c38c38ca4f None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:05.757315 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d395d-dcba-4e3e-8a30-6cffebae4a33 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:05.758919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:05.772627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae31d7-642d-4ee8-a0d4-62440dfc4c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:05.773320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae31d7-642d-4ee8-a0d4-62440dfc4c2e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:05.778929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae31d7-642d-4ee8-a0d4-62440dfc4c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:05.778929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae31d7-642d-4ee8-a0d4-62440dfc4c2e None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:05.778929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae31d7-642d-4ee8-a0d4-62440dfc4c2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:05.778929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae31d7-642d-4ee8-a0d4-62440dfc4c2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:05.778929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae31d7-642d-4ee8-a0d4-62440dfc4c2e None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:05.795091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 125/264] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:05.802204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae31d7-642d-4ee8-a0d4-62440dfc4c2e None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:05.804524 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416619fe-8670-4fb3-8409-4998a3206ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:05.805458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416619fe-8670-4fb3-8409-4998a3206ac0 None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:05.805743 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416619fe-8670-4fb3-8409-4998a3206ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:05.806022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416619fe-8670-4fb3-8409-4998a3206ac0 None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f/users/8a1fb61c0f724950be189514be7e1253/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:05.810615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:05.811093 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416619fe-8670-4fb3-8409-4998a3206ac0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b271a003871a42fb99fa8b9ca8f9017f, user_id=8a1fb61c0f724950be189514be7e1253, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:05.812608 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416619fe-8670-4fb3-8409-4998a3206ac0 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:05.907566 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 140/265] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:12:05 2022] PUT /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f/users/8a1fb61c0f724950be189514be7e1253/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:12:05.909272 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69e019-dbdd-4694-a20e-22e53a1cadfa None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:05.909791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69e019-dbdd-4694-a20e-22e53a1cadfa None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:05.910278 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69e019-dbdd-4694-a20e-22e53a1cadfa None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:05.913645 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 141/266] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:05.919612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475542cd-aa17-4cbb-9634-790cf9cda18d None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:05.920211 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475542cd-aa17-4cbb-9634-790cf9cda18d None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:05.920693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475542cd-aa17-4cbb-9634-790cf9cda18d None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:05.923165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 142/267] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:05.924655 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a647528-9fd1-4e1b-a45c-4429ce57bd18 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:05.928542 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a647528-9fd1-4e1b-a45c-4429ce57bd18 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:05.929061 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a647528-9fd1-4e1b-a45c-4429ce57bd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:05.949430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-2a647528-9fd1-4e1b-a45c-4429ce57bd18 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:05.950858 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2a647528-9fd1-4e1b-a45c-4429ce57bd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:05.954364 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae31d7-642d-4ee8-a0d4-62440dfc4c2e None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:06.006028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf721c00-355b-44c8-b0b6-8d0358a6fd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:06.009222 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721c00-355b-44c8-b0b6-8d0358a6fd7b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:06.010644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721c00-355b-44c8-b0b6-8d0358a6fd7b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:06.011132 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721c00-355b-44c8-b0b6-8d0358a6fd7b None admin] PATH_INFO: `/v3/users/ab6e3ab00f4c4e96870913ffd50b64f9` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:06.012752 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf721c00-355b-44c8-b0b6-8d0358a6fd7b None admin] RBAC: Authorizing `identity:get_user(user_id=ab6e3ab00f4c4e96870913ffd50b64f9)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:06.015618 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf721c00-355b-44c8-b0b6-8d0358a6fd7b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:06.018307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 126/268] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:12:05 2022] GET /identity/v3/users/ab6e3ab00f4c4e96870913ffd50b64f9 => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:06.020574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1571465-8f5e-42bc-909d-be247ed533c9 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:06.021174 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1571465-8f5e-42bc-909d-be247ed533c9 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:06.021633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1571465-8f5e-42bc-909d-be247ed533c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:06.045666 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-f1571465-8f5e-42bc-909d-be247ed533c9 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:06.047306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f1571465-8f5e-42bc-909d-be247ed533c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:06.049344 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 143/269] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:06.050455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a647528-9fd1-4e1b-a45c-4429ce57bd18 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:06.145468 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:06.150295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 127/270] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:06.151021 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1571465-8f5e-42bc-909d-be247ed533c9 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:06.180367 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:06.184444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a647528-9fd1-4e1b-a45c-4429ce57bd18 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:06.235625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1dd8d-2fb0-4da5-90c1-5ca4f52dffc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:06.237221 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1dd8d-2fb0-4da5-90c1-5ca4f52dffc4 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:06.237520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1dd8d-2fb0-4da5-90c1-5ca4f52dffc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:06.237803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1dd8d-2fb0-4da5-90c1-5ca4f52dffc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:06.268959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1571465-8f5e-42bc-909d-be247ed533c9 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:06.305974 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab62028-9c9f-4326-a0a8-5175b2976da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:06.306854 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62028-9c9f-4326-a0a8-5175b2976da3 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:06.307146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62028-9c9f-4326-a0a8-5175b2976da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:06.307426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62028-9c9f-4326-a0a8-5175b2976da3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:06.308623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab62028-9c9f-4326-a0a8-5175b2976da3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:06.310067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab62028-9c9f-4326-a0a8-5175b2976da3 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:06.311748 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 128/271] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:06.312310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1dd8d-2fb0-4da5-90c1-5ca4f52dffc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:06.312580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab62028-9c9f-4326-a0a8-5175b2976da3 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:06.313922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1dd8d-2fb0-4da5-90c1-5ca4f52dffc4 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:06.383611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:06.408957 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:06.420019 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab62028-9c9f-4326-a0a8-5175b2976da3 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:06.420514 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 144/272] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:06.421509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1dd8d-2fb0-4da5-90c1-5ca4f52dffc4 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:06.461086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc247158-c116-4a14-9835-ff0bedbea618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:06.461953 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc247158-c116-4a14-9835-ff0bedbea618 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:06.462247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc247158-c116-4a14-9835-ff0bedbea618 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:06.462555 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc247158-c116-4a14-9835-ff0bedbea618 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:06.473815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc247158-c116-4a14-9835-ff0bedbea618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:06.482869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-fc247158-c116-4a14-9835-ff0bedbea618 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:06.482869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:06.482869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:06.482869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:06.482869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:06.482869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:06.482869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:06.482869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:06.482869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:06.482869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:06.482869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:06.482869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:06.482869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:06.482869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:06.482869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:06.482869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:06.483762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:06.483762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:06.483762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:06.483762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:06.483762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:06.483762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:06.483762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:06.483762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:06.483762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:06.483762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:06.483762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:06.483762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:06.483762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:06.483762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:06.483762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:06.483762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:06.484583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:06.484583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:06.484583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:06.484583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:06.484583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:06.484583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:06.484583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:06.484583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:06.484583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:06.484583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:06.484583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:06.484583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:06.484583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 129/273] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:06.485249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc247158-c116-4a14-9835-ff0bedbea618 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:06.536342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1dd8d-2fb0-4da5-90c1-5ca4f52dffc4 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:06.572817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba948c0-9bcb-4b47-aebc-7ed36c46a7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:06.573492 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba948c0-9bcb-4b47-aebc-7ed36c46a7b7 None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:06.573776 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba948c0-9bcb-4b47-aebc-7ed36c46a7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:06.574057 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba948c0-9bcb-4b47-aebc-7ed36c46a7b7 None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:06.574795 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba948c0-9bcb-4b47-aebc-7ed36c46a7b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:06.576250 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba948c0-9bcb-4b47-aebc-7ed36c46a7b7 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:06.594643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc247158-c116-4a14-9835-ff0bedbea618 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:06.620865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:06.628766 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557ef49-ed97-4cb5-93ad-27e4f7febabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:06.629891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557ef49-ed97-4cb5-93ad-27e4f7febabb None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:06.630173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557ef49-ed97-4cb5-93ad-27e4f7febabb None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:06.631160 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557ef49-ed97-4cb5-93ad-27e4f7febabb None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:06.632341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557ef49-ed97-4cb5-93ad-27e4f7febabb None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:06.633789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557ef49-ed97-4cb5-93ad-27e4f7febabb None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:06.635695 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 130/274] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:06 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:06.636856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e4591-3cb7-4f1f-b5c0-69bfd71e0815 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:06.637148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e4591-3cb7-4f1f-b5c0-69bfd71e0815 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:06.637427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e4591-3cb7-4f1f-b5c0-69bfd71e0815 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:06.642762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 131/275] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:06.642762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e4591-3cb7-4f1f-b5c0-69bfd71e0815 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:06.704479 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 145/276] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:12:06 2022] POST /identity/v3/users => generated 293 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:12:06.705332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba948c0-9bcb-4b47-aebc-7ed36c46a7b7 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:06.760169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e4591-3cb7-4f1f-b5c0-69bfd71e0815 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:06.780748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:06.804063 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b93ac-94ae-429a-a23b-b84f3ff291f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:06.805080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b93ac-94ae-429a-a23b-b84f3ff291f8 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:06.805588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b93ac-94ae-429a-a23b-b84f3ff291f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:06.806043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b93ac-94ae-429a-a23b-b84f3ff291f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:06.807009 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b93ac-94ae-429a-a23b-b84f3ff291f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:06.807496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b93ac-94ae-429a-a23b-b84f3ff291f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:06.809118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b93ac-94ae-429a-a23b-b84f3ff291f8 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:06.821355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 132/277] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:12:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:06.823110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0afbe-e33d-4c32-9587-d0d9ee279e75 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:06.828800 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0afbe-e33d-4c32-9587-d0d9ee279e75 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:06.828800 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0afbe-e33d-4c32-9587-d0d9ee279e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:06.842290 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-5ae0afbe-e33d-4c32-9587-d0d9ee279e75 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:06.843949 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5ae0afbe-e33d-4c32-9587-d0d9ee279e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:06.845312 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba948c0-9bcb-4b47-aebc-7ed36c46a7b7 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:06.905301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f18cf-79ed-42a1-9859-11553db4414e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:06.906151 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f18cf-79ed-42a1-9859-11553db4414e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:06.906560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f18cf-79ed-42a1-9859-11553db4414e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:06.906931 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f18cf-79ed-42a1-9859-11553db4414e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:06.907868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f18cf-79ed-42a1-9859-11553db4414e None admin] RBAC: Adding query filter params (role.id=2da8d544efe44ce6a41160d5759be402, scope.project.id=b271a003871a42fb99fa8b9ca8f9017f, user.id=ab6e3ab00f4c4e96870913ffd50b64f9) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:06.908294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f18cf-79ed-42a1-9859-11553db4414e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:06.909850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f18cf-79ed-42a1-9859-11553db4414e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:06.916635 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 146/278] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:12:06 2022] GET /identity/v3/role_assignments?user.id=ab6e3ab00f4c4e96870913ffd50b64f9&scope.project.id=b271a003871a42fb99fa8b9ca8f9017f&role.id=2da8d544efe44ce6a41160d5759be402 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:06.917100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:06.917646 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f18cf-79ed-42a1-9859-11553db4414e None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:06.941001 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 133/279] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:06.966830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0afbe-e33d-4c32-9587-d0d9ee279e75 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:07.046406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f18cf-79ed-42a1-9859-11553db4414e None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:07.061620 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:07.086281 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac16a4d-accc-4da3-82d2-a1bd5aaa94e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:07.087421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac16a4d-accc-4da3-82d2-a1bd5aaa94e4 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:07.087888 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac16a4d-accc-4da3-82d2-a1bd5aaa94e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:07.088285 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac16a4d-accc-4da3-82d2-a1bd5aaa94e4 None admin] PATH_INFO: `/v3/users/c572a3809e76418a9879c801a3feeb13` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:07.097544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0afbe-e33d-4c32-9587-d0d9ee279e75 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:07.126075 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac16a4d-accc-4da3-82d2-a1bd5aaa94e4 None admin] RBAC: Authorizing `identity:get_user(user_id=c572a3809e76418a9879c801a3feeb13)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:07.127688 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac16a4d-accc-4da3-82d2-a1bd5aaa94e4 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:07.129810 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 147/280] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:12:06 2022] GET /identity/v3/users/c572a3809e76418a9879c801a3feeb13 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:07.136422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac16a4d-accc-4da3-82d2-a1bd5aaa94e4 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:07.147804 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9d328-e3f0-4493-b72f-44f89f807f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:07.149262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9d328-e3f0-4493-b72f-44f89f807f4f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:07.149558 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9d328-e3f0-4493-b72f-44f89f807f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:07.149839 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9d328-e3f0-4493-b72f-44f89f807f4f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:07.167128 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a9d328-e3f0-4493-b72f-44f89f807f4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:07.177358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-e5a9d328-e3f0-4493-b72f-44f89f807f4f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:12:07.177358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:07.177358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:07.177358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:07.177358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:07.177358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:07.177358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:07.177358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:07.177358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:07.177358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:07.177358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:07.177358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:07.177358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:07.177358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:07.177358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:07.177358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:07.178363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:07.178363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:07.178363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:07.178363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:07.178363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:07.178363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:07.178363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:07.178363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:07.178363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:07.178363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:07.178363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:07.178363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:07.178363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:07.178363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:07.178363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:07.178363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:07.179168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:07.179168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:07.179168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:07.179168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:07.179168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:07.179168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:07.179168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:07.179168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:07.179168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:07.179168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:07.179168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:12:07.179168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:07.180470 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 134/281] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:07.188782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9d328-e3f0-4493-b72f-44f89f807f4f None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:07.253743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:07.281912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac16a4d-accc-4da3-82d2-a1bd5aaa94e4 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:07.323224 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9d328-e3f0-4493-b72f-44f89f807f4f None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:07.325571 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432d4d7-ce7c-40b4-b4b8-aeddaf82b132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:07.326247 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432d4d7-ce7c-40b4-b4b8-aeddaf82b132 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:07.326577 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432d4d7-ce7c-40b4-b4b8-aeddaf82b132 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:07.326861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432d4d7-ce7c-40b4-b4b8-aeddaf82b132 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:07.327707 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432d4d7-ce7c-40b4-b4b8-aeddaf82b132 None admin] RBAC: Adding query filter params (role.id=d85783606f6649f5b9465f407cafb610, user.id=c572a3809e76418a9879c801a3feeb13, scope.system=all) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:07.328092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432d4d7-ce7c-40b4-b4b8-aeddaf82b132 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:07.329684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432d4d7-ce7c-40b4-b4b8-aeddaf82b132 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:07.336168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 148/282] 10.222.0.78 () {56 vars in 1308 bytes} [Wed Jul 20 15:12:07 2022] GET /identity/v3/role_assignments?user.id=c572a3809e76418a9879c801a3feeb13&scope.system=all&role.id=d85783606f6649f5b9465f407cafb610 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:07.366123 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d06a45-4ceb-4d7f-b4e9-4f7036e6ed15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:07.366855 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d06a45-4ceb-4d7f-b4e9-4f7036e6ed15 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:07.367298 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d06a45-4ceb-4d7f-b4e9-4f7036e6ed15 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:07.367589 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d06a45-4ceb-4d7f-b4e9-4f7036e6ed15 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:07.368366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d06a45-4ceb-4d7f-b4e9-4f7036e6ed15 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:07.368680 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d06a45-4ceb-4d7f-b4e9-4f7036e6ed15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:07.370118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d06a45-4ceb-4d7f-b4e9-4f7036e6ed15 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:07.380571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 135/283] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:12:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:07.386745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432d4d7-ce7c-40b4-b4b8-aeddaf82b132 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:07.437398 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:07.535506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432d4d7-ce7c-40b4-b4b8-aeddaf82b132 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:07.570015 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631236c-5c98-4d07-9aac-88fbf2c68d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:07.570919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631236c-5c98-4d07-9aac-88fbf2c68d51 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:07.571199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631236c-5c98-4d07-9aac-88fbf2c68d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:07.571472 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631236c-5c98-4d07-9aac-88fbf2c68d51 None admin] PATH_INFO: `/v3/users/770a31c527274941b3df3641eafdfe24` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:07.572681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631236c-5c98-4d07-9aac-88fbf2c68d51 None admin] RBAC: Authorizing `identity:get_user(user_id=770a31c527274941b3df3641eafdfe24)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:07.574185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631236c-5c98-4d07-9aac-88fbf2c68d51 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:07.575960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 149/284] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:12:07 2022] GET /identity/v3/users/770a31c527274941b3df3641eafdfe24 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:07.581336 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d06a45-4ceb-4d7f-b4e9-4f7036e6ed15 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:07.722013 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d06a45-4ceb-4d7f-b4e9-4f7036e6ed15 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:07.736406 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:07.770247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb02393-f1e8-4a2f-9368-c3a77c913fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:07.771176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb02393-f1e8-4a2f-9368-c3a77c913fad None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:07.771459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb02393-f1e8-4a2f-9368-c3a77c913fad None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:07.771740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb02393-f1e8-4a2f-9368-c3a77c913fad None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:07.779040 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb02393-f1e8-4a2f-9368-c3a77c913fad None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:07.780515 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb02393-f1e8-4a2f-9368-c3a77c913fad None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:07.782648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 136/285] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:07 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:07.788112 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631236c-5c98-4d07-9aac-88fbf2c68d51 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:07.939301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631236c-5c98-4d07-9aac-88fbf2c68d51 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:07.940722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:07.986026 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b80ce-946f-40cf-940d-008cef74102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:07.987025 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b80ce-946f-40cf-940d-008cef74102e None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:07.987308 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b80ce-946f-40cf-940d-008cef74102e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:07.987584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b80ce-946f-40cf-940d-008cef74102e None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f/users/770a31c527274941b3df3641eafdfe24/roles/6c39460198da48b7b86f53b577c698b1` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:07.995733 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b80ce-946f-40cf-940d-008cef74102e None admin] RBAC: Authorizing `identity:create_grant(project_id=b271a003871a42fb99fa8b9ca8f9017f, user_id=770a31c527274941b3df3641eafdfe24, role_id=6c39460198da48b7b86f53b577c698b1)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:07.997214 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b80ce-946f-40cf-940d-008cef74102e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:08.081470 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 150/286] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:12:07 2022] PUT /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f/users/770a31c527274941b3df3641eafdfe24/roles/6c39460198da48b7b86f53b577c698b1 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:12:08.442487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044ca0a-705b-44a9-8485-3d33c6fcbfce None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:08.443973 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044ca0a-705b-44a9-8485-3d33c6fcbfce None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:08.444462 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044ca0a-705b-44a9-8485-3d33c6fcbfce None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:08.446288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 137/287] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:08.452448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55262325-e3c7-4438-878e-16b1da34e977 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:08.453405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55262325-e3c7-4438-878e-16b1da34e977 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:08.453916 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55262325-e3c7-4438-878e-16b1da34e977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:08.474370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-55262325-e3c7-4438-878e-16b1da34e977 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:08.475894 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-55262325-e3c7-4438-878e-16b1da34e977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:08.571303 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 151/288] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:08.605402 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044ca0a-705b-44a9-8485-3d33c6fcbfce None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:08.729105 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044ca0a-705b-44a9-8485-3d33c6fcbfce None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:08.764249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f8a96-2158-45c5-a371-6586b8d2f0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:08.765047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f8a96-2158-45c5-a371-6586b8d2f0dd None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:08.765327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f8a96-2158-45c5-a371-6586b8d2f0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:08.765599 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f8a96-2158-45c5-a371-6586b8d2f0dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:08.775913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f8a96-2158-45c5-a371-6586b8d2f0dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:08.783643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-e00f8a96-2158-45c5-a371-6586b8d2f0dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:08.783643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:08.783643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:08.783643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:08.783643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:08.783643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:08.783643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:08.783643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:08.783643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:08.783643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:08.783643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:08.783643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:08.783643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:08.783643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:08.783643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:08.783643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:08.784644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:08.784644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:08.784644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:08.784644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:08.784644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:08.784644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:08.784644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:08.784644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:08.784644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:08.784644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:08.784644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:08.784644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:08.784644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:08.784644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:08.784644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:08.784644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:08.785464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:08.785464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:08.785464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:08.785464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:08.785464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:08.785464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:08.785464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:08.785464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:08.785464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:08.785464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:08.785464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:08.785464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:08.785464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 138/289] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:08.795473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55262325-e3c7-4438-878e-16b1da34e977 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:08.907128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:08.916334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55262325-e3c7-4438-878e-16b1da34e977 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:08.956863 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cd227-d734-4f9b-9b55-b2e35870115d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:08.957548 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cd227-d734-4f9b-9b55-b2e35870115d None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:08.957831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cd227-d734-4f9b-9b55-b2e35870115d None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:08.958111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cd227-d734-4f9b-9b55-b2e35870115d None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:08.958927 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13cd227-d734-4f9b-9b55-b2e35870115d None admin] RBAC: Adding query filter params (name=member) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:08.959240 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13cd227-d734-4f9b-9b55-b2e35870115d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:08.960626 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13cd227-d734-4f9b-9b55-b2e35870115d None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:08.969360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 152/290] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:12:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:08.975746 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f8a96-2158-45c5-a371-6586b8d2f0dd None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:09.034932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:09.118826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f8a96-2158-45c5-a371-6586b8d2f0dd None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:09.171009 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d43d3-2650-4f8f-9e26-d4c55b321734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:09.171911 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d43d3-2650-4f8f-9e26-d4c55b321734 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:09.172198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d43d3-2650-4f8f-9e26-d4c55b321734 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:09.172477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d43d3-2650-4f8f-9e26-d4c55b321734 None admin] PATH_INFO: `/v3/users/8a1fb61c0f724950be189514be7e1253` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:09.173757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d43d3-2650-4f8f-9e26-d4c55b321734 None admin] RBAC: Authorizing `identity:get_user(user_id=8a1fb61c0f724950be189514be7e1253)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:09.175237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d43d3-2650-4f8f-9e26-d4c55b321734 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:09.177865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 139/291] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:12:08 2022] GET /identity/v3/users/8a1fb61c0f724950be189514be7e1253 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:09.183106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cd227-d734-4f9b-9b55-b2e35870115d None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:09.302450 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cd227-d734-4f9b-9b55-b2e35870115d None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:09.345785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3d1b0-f104-4a78-8a9d-3ebc5f879667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:09.346475 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3d1b0-f104-4a78-8a9d-3ebc5f879667 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:09.346766 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3d1b0-f104-4a78-8a9d-3ebc5f879667 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:09.347042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3d1b0-f104-4a78-8a9d-3ebc5f879667 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:09.377305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:09.437799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d3d1b0-f104-4a78-8a9d-3ebc5f879667 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:09.439312 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d3d1b0-f104-4a78-8a9d-3ebc5f879667 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:09.577994 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 153/292] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:09.584544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d43d3-2650-4f8f-9e26-d4c55b321734 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:09.594771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b4abb-1523-4749-af95-3fefad7db0fa None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:09.595058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b4abb-1523-4749-af95-3fefad7db0fa None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:09.595338 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b4abb-1523-4749-af95-3fefad7db0fa None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:09.597015 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 154/293] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:12:09.607196 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de9a6d-1833-46f8-bdb8-69503892ff5c None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:09.607483 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de9a6d-1833-46f8-bdb8-69503892ff5c None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:09.607761 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de9a6d-1833-46f8-bdb8-69503892ff5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:09.639219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-11de9a6d-1833-46f8-bdb8-69503892ff5c None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:09.640412 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-11de9a6d-1833-46f8-bdb8-69503892ff5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:09.727684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:09.745680 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d43d3-2650-4f8f-9e26-d4c55b321734 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:09.746432 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 155/294] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:12:09.747912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4199bcb-7846-4849-a1ee-491546a4acc0 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:09.748300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4199bcb-7846-4849-a1ee-491546a4acc0 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:09.748589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4199bcb-7846-4849-a1ee-491546a4acc0 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:09.750118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 156/295] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:09.759127 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:09.768150 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b97dd8-8afc-453c-bb9c-12fffd1358b3 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:09.768484 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b97dd8-8afc-453c-bb9c-12fffd1358b3 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:09.768780 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b97dd8-8afc-453c-bb9c-12fffd1358b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:09.785514 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b70fb-a35f-449b-9ee3-e029c6909145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:09.786472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b70fb-a35f-449b-9ee3-e029c6909145 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:09.786761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b70fb-a35f-449b-9ee3-e029c6909145 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:09.787035 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b70fb-a35f-449b-9ee3-e029c6909145 None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:09.788312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b70fb-a35f-449b-9ee3-e029c6909145 None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:09.789203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-e3b97dd8-8afc-453c-bb9c-12fffd1358b3 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:09.789693 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b70fb-a35f-449b-9ee3-e029c6909145 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:09.790440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e3b97dd8-8afc-453c-bb9c-12fffd1358b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:09.791739 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 140/296] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:09 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:09.792665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b70fb-a35f-449b-9ee3-e029c6909145 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:09.859670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 157/297] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:12:09.863463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b97dd8-8afc-453c-bb9c-12fffd1358b3 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:09.903474 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b70fb-a35f-449b-9ee3-e029c6909145 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:09.939554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e943c6b-8d48-42d6-88cb-bd3a26c06162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:09.940340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e943c6b-8d48-42d6-88cb-bd3a26c06162 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:09.940627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e943c6b-8d48-42d6-88cb-bd3a26c06162 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:09.940902 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e943c6b-8d48-42d6-88cb-bd3a26c06162 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:09.942057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e943c6b-8d48-42d6-88cb-bd3a26c06162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:09.944081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e943c6b-8d48-42d6-88cb-bd3a26c06162 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:09.945674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 141/298] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:09.946851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0328f0-bd15-4f18-aab8-c02366641f22 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:09.947133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0328f0-bd15-4f18-aab8-c02366641f22 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:09.947406 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0328f0-bd15-4f18-aab8-c02366641f22 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:09.950950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 142/299] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:09.950950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0328f0-bd15-4f18-aab8-c02366641f22 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:09.976323 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b97dd8-8afc-453c-bb9c-12fffd1358b3 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:09.992506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:09.996895 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:10.015518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05203e-77d2-49de-9a3d-32c4b5ef3b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:10.016170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05203e-77d2-49de-9a3d-32c4b5ef3b48 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:10.016451 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05203e-77d2-49de-9a3d-32c4b5ef3b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:10.016732 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05203e-77d2-49de-9a3d-32c4b5ef3b48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:10.017549 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05203e-77d2-49de-9a3d-32c4b5ef3b48 None admin] RBAC: Adding query filter params (role.id=d85783606f6649f5b9465f407cafb610, scope.project.id=b271a003871a42fb99fa8b9ca8f9017f, user.id=8a1fb61c0f724950be189514be7e1253) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:10.017858 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05203e-77d2-49de-9a3d-32c4b5ef3b48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:10.019312 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05203e-77d2-49de-9a3d-32c4b5ef3b48 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:10.024995 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 158/300] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:12:09 2022] GET /identity/v3/role_assignments?user.id=8a1fb61c0f724950be189514be7e1253&scope.project.id=b271a003871a42fb99fa8b9ca8f9017f&role.id=d85783606f6649f5b9465f407cafb610 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:10.025805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05203e-77d2-49de-9a3d-32c4b5ef3b48 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:10.070937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0328f0-bd15-4f18-aab8-c02366641f22 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:10.111484 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee7b63-680a-430d-8fe1-3c3433094e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:10.112334 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee7b63-680a-430d-8fe1-3c3433094e4f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:10.112729 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee7b63-680a-430d-8fe1-3c3433094e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:10.113064 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee7b63-680a-430d-8fe1-3c3433094e4f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:10.129757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee7b63-680a-430d-8fe1-3c3433094e4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:10.141048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-c0ee7b63-680a-430d-8fe1-3c3433094e4f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:12:10.141048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:10.141048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:10.141048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:10.141048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:10.141048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:10.141048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:10.141048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:10.141048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:10.141048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:10.141048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:10.141048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:10.141048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:10.141048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:10.141048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:10.141048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:10.142031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:10.142031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:10.142031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:10.142031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:10.142031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:10.142031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:10.142031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:10.142031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:10.142031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:10.142031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:10.142031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:10.142031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:10.142031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:10.142031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:10.142031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:10.142031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:10.142861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:10.142861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:10.142861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:10.142861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:10.142861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:10.142861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:10.142861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:10.142861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:10.142861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:10.142861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:10.142861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:12:10.142861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:10.146440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 143/301] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:10.148828 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60504ab2-e78f-4121-bfc5-46d5dd7f394a None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:10.149127 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60504ab2-e78f-4121-bfc5-46d5dd7f394a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:10.149409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60504ab2-e78f-4121-bfc5-46d5dd7f394a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:10.168981 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-60504ab2-e78f-4121-bfc5-46d5dd7f394a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:10.170193 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-60504ab2-e78f-4121-bfc5-46d5dd7f394a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:10.171339 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05203e-77d2-49de-9a3d-32c4b5ef3b48 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:10.216945 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f1902-ce23-4ecd-87fc-e470a8a6ecd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:10.216945 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f1902-ce23-4ecd-87fc-e470a8a6ecd1 None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:10.216945 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f1902-ce23-4ecd-87fc-e470a8a6ecd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:10.216945 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f1902-ce23-4ecd-87fc-e470a8a6ecd1 None admin] PATH_INFO: `/v3/groups` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:10.217589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f1902-ce23-4ecd-87fc-e470a8a6ecd1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:10.219235 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f1902-ce23-4ecd-87fc-e470a8a6ecd1 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:10.259070 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 144/302] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:10.259929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60504ab2-e78f-4121-bfc5-46d5dd7f394a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:10.295056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:10.321210 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 159/303] 10.222.0.78 () {60 vars in 1134 bytes} [Wed Jul 20 15:12:10 2022] POST /identity/v3/groups => generated 229 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:12:10.322330 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f1902-ce23-4ecd-87fc-e470a8a6ecd1 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:10.371732 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60504ab2-e78f-4121-bfc5-46d5dd7f394a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:10.384545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:10.415846 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83f1e8-cdd2-4029-9156-c3c2542f1afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:10.416583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83f1e8-cdd2-4029-9156-c3c2542f1afb None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:10.416873 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83f1e8-cdd2-4029-9156-c3c2542f1afb None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:10.417150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83f1e8-cdd2-4029-9156-c3c2542f1afb None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:10.417922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83f1e8-cdd2-4029-9156-c3c2542f1afb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:10.418234 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83f1e8-cdd2-4029-9156-c3c2542f1afb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:10.419697 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83f1e8-cdd2-4029-9156-c3c2542f1afb None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:10.431219 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 145/304] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:12:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:10.436958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83f1e8-cdd2-4029-9156-c3c2542f1afb None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:10.464754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f1902-ce23-4ecd-87fc-e470a8a6ecd1 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:10.503944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91715518-326d-48cf-8ca5-3c4051656ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:10.504803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91715518-326d-48cf-8ca5-3c4051656ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:10.505144 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91715518-326d-48cf-8ca5-3c4051656ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:10.505947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91715518-326d-48cf-8ca5-3c4051656ba8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:10.521061 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91715518-326d-48cf-8ca5-3c4051656ba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:10.529759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-91715518-326d-48cf-8ca5-3c4051656ba8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:10.529759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:10.529759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:10.529759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:10.529759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:10.529759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:10.529759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:10.529759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:10.529759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:10.529759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:10.529759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:10.529759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:10.529759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:10.529759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:10.529759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:10.529759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:10.530702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:10.530702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:10.530702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:10.530702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:10.530702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:10.530702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:10.530702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:10.530702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:10.530702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:10.530702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:10.530702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:10.530702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:10.530702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:10.530702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:10.530702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:10.530702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:10.531520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:10.531520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:10.531520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:10.531520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:10.531520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:10.531520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:10.531520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:10.531520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:10.531520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:10.531520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:10.531520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:10.531520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:10.532600 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 160/305] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:10.539110 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91715518-326d-48cf-8ca5-3c4051656ba8 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:10.552901 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:10.562946 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83f1e8-cdd2-4029-9156-c3c2542f1afb None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:10.610936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0c64c-6d45-4f2f-b2ab-d167f1fa8bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:10.611812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0c64c-6d45-4f2f-b2ab-d167f1fa8bfa None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:10.612699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0c64c-6d45-4f2f-b2ab-d167f1fa8bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:10.612981 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0c64c-6d45-4f2f-b2ab-d167f1fa8bfa None admin] PATH_INFO: `/v3/users/8ba7526b3e9540febcdb59dc597181a8` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:10.631388 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d0c64c-6d45-4f2f-b2ab-d167f1fa8bfa None admin] RBAC: Authorizing `identity:get_user(user_id=8ba7526b3e9540febcdb59dc597181a8)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:10.633075 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d0c64c-6d45-4f2f-b2ab-d167f1fa8bfa None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:10.635301 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 146/306] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:12:10 2022] GET /identity/v3/users/8ba7526b3e9540febcdb59dc597181a8 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:10.640675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0c64c-6d45-4f2f-b2ab-d167f1fa8bfa None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:10.657010 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:10.660780 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91715518-326d-48cf-8ca5-3c4051656ba8 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:10.697771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e8fc3-d53f-4960-bb94-b72c28740ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:10.698666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e8fc3-d53f-4960-bb94-b72c28740ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:10.699052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e8fc3-d53f-4960-bb94-b72c28740ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:10.699463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e8fc3-d53f-4960-bb94-b72c28740ac0 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:10.700317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e8fc3-d53f-4960-bb94-b72c28740ac0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:10.700697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e8fc3-d53f-4960-bb94-b72c28740ac0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:10.702291 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e8fc3-d53f-4960-bb94-b72c28740ac0 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:10.715882 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 161/307] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:12:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:10.716952 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e8fc3-d53f-4960-bb94-b72c28740ac0 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:10.749176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0c64c-6d45-4f2f-b2ab-d167f1fa8bfa None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:10.785335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2c1b0-84a8-41b6-86e8-f5a20cc3a15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:10.785993 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2c1b0-84a8-41b6-86e8-f5a20cc3a15f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:10.786301 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2c1b0-84a8-41b6-86e8-f5a20cc3a15f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:10.786608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2c1b0-84a8-41b6-86e8-f5a20cc3a15f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:10.787435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2c1b0-84a8-41b6-86e8-f5a20cc3a15f None admin] RBAC: Adding query filter params (role.id=6c39460198da48b7b86f53b577c698b1, user.id=8ba7526b3e9540febcdb59dc597181a8, scope.system=all) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:10.787948 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2c1b0-84a8-41b6-86e8-f5a20cc3a15f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:10.789377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2c1b0-84a8-41b6-86e8-f5a20cc3a15f None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:10.802638 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 147/308] 10.222.0.78 () {56 vars in 1308 bytes} [Wed Jul 20 15:12:10 2022] GET /identity/v3/role_assignments?user.id=8ba7526b3e9540febcdb59dc597181a8&scope.system=all&role.id=6c39460198da48b7b86f53b577c698b1 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:12:10.802638 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c39f1-c517-40b8-a024-38af1fd4fa26 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:10.802638 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c39f1-c517-40b8-a024-38af1fd4fa26 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:10.802638 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c39f1-c517-40b8-a024-38af1fd4fa26 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:10.802638 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 148/309] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:10.805063 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1067d09-baa4-43b4-9b97-9ac383c367f8 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:10.805356 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1067d09-baa4-43b4-9b97-9ac383c367f8 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:10.805637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1067d09-baa4-43b4-9b97-9ac383c367f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:10.824630 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-f1067d09-baa4-43b4-9b97-9ac383c367f8 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:10.825897 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f1067d09-baa4-43b4-9b97-9ac383c367f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:10.835730 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e8fc3-d53f-4960-bb94-b72c28740ac0 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:10.839130 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:10.879492 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c1183-7d77-4cbf-9d50-e0089dc5e327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:10.881462 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c1183-7d77-4cbf-9d50-e0089dc5e327 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:10.881972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c1183-7d77-4cbf-9d50-e0089dc5e327 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:10.882282 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c1183-7d77-4cbf-9d50-e0089dc5e327 None admin] PATH_INFO: `/v3/users/c572a3809e76418a9879c801a3feeb13` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:10.883864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c1183-7d77-4cbf-9d50-e0089dc5e327 None admin] RBAC: Authorizing `identity:get_user(user_id=c572a3809e76418a9879c801a3feeb13)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:10.886314 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c1183-7d77-4cbf-9d50-e0089dc5e327 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:10.889008 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 162/310] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:12:10 2022] GET /identity/v3/users/c572a3809e76418a9879c801a3feeb13 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:12:10.895272 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c1183-7d77-4cbf-9d50-e0089dc5e327 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:10.914793 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 149/311] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:12:10.930469 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:10.934563 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1067d09-baa4-43b4-9b97-9ac383c367f8 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:11.020689 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c1183-7d77-4cbf-9d50-e0089dc5e327 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:11.051099 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1067d09-baa4-43b4-9b97-9ac383c367f8 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:11.056113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddb56b-b732-401a-9f2b-2262664cc39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:11.057010 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddb56b-b732-401a-9f2b-2262664cc39c None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:11.057294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddb56b-b732-401a-9f2b-2262664cc39c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:11.057578 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddb56b-b732-401a-9f2b-2262664cc39c None admin] PATH_INFO: `/v3/system/users/c572a3809e76418a9879c801a3feeb13/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:11.059143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddb56b-b732-401a-9f2b-2262664cc39c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c572a3809e76418a9879c801a3feeb13, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:11.060546 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddb56b-b732-401a-9f2b-2262664cc39c None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:11.095423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d66a3e-66ee-444c-882a-7579d11aee31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:11.096314 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d66a3e-66ee-444c-882a-7579d11aee31 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:11.096625 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d66a3e-66ee-444c-882a-7579d11aee31 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:11.096961 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d66a3e-66ee-444c-882a-7579d11aee31 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:11.108282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d66a3e-66ee-444c-882a-7579d11aee31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:11.117227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-19d66a3e-66ee-444c-882a-7579d11aee31 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:12:11.117227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:11.117227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:11.117227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:11.117227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:11.117227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:11.117227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:11.117227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:11.117227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:11.117227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:11.117227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:11.117227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:11.117227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:11.117227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:11.117227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:11.117227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:11.118156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:11.118156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:11.118156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:11.118156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:11.118156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:11.118156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:11.118156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:11.118156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:11.118156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:11.118156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:11.118156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:11.118156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:11.118156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:11.118156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:11.118156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:11.118156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:11.119435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:11.119435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:11.119435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:11.119435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:11.119435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:11.119435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:11.119435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:11.119435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:11.119435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:11.119435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:11.119435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:12:11.119435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:11.119435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 150/312] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:11.127533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d66a3e-66ee-444c-882a-7579d11aee31 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:11.138648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:11.212113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 163/313] 10.222.0.78 () {58 vars in 1334 bytes} [Wed Jul 20 15:12:10 2022] PUT /identity/v3/system/users/c572a3809e76418a9879c801a3feeb13/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:12:11.243417 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d66a3e-66ee-444c-882a-7579d11aee31 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:11.281651 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117a9cf-df05-4c57-a7e8-3761a89eaa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:11.282543 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117a9cf-df05-4c57-a7e8-3761a89eaa2d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:11.282931 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117a9cf-df05-4c57-a7e8-3761a89eaa2d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:11.283212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117a9cf-df05-4c57-a7e8-3761a89eaa2d None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:11.284003 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117a9cf-df05-4c57-a7e8-3761a89eaa2d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:11.284317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117a9cf-df05-4c57-a7e8-3761a89eaa2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:11.285882 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117a9cf-df05-4c57-a7e8-3761a89eaa2d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:11.296332 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 151/314] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:12:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:12:11.303184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddb56b-b732-401a-9f2b-2262664cc39c None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:11.403321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:11.412925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddb56b-b732-401a-9f2b-2262664cc39c None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:11.451061 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029efda7-5c62-422a-82ce-cb53b7ac6c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:11.451990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029efda7-5c62-422a-82ce-cb53b7ac6c97 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:11.452279 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029efda7-5c62-422a-82ce-cb53b7ac6c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:11.452560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029efda7-5c62-422a-82ce-cb53b7ac6c97 None admin] PATH_INFO: `/v3/users/770a31c527274941b3df3641eafdfe24` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:11.453800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029efda7-5c62-422a-82ce-cb53b7ac6c97 None admin] RBAC: Authorizing `identity:get_user(user_id=770a31c527274941b3df3641eafdfe24)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:11.455358 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029efda7-5c62-422a-82ce-cb53b7ac6c97 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:11.457299 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 164/315] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:12:11 2022] GET /identity/v3/users/770a31c527274941b3df3641eafdfe24 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:11.463533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117a9cf-df05-4c57-a7e8-3761a89eaa2d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:11.535952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:11.575240 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117a9cf-df05-4c57-a7e8-3761a89eaa2d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:11.611814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7664d6a-3103-4215-aa62-e00ca5646362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:11.613350 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7664d6a-3103-4215-aa62-e00ca5646362 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:11.613877 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7664d6a-3103-4215-aa62-e00ca5646362 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:11.614379 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7664d6a-3103-4215-aa62-e00ca5646362 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:11.686229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7664d6a-3103-4215-aa62-e00ca5646362 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:11.688094 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7664d6a-3103-4215-aa62-e00ca5646362 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:11.759658 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:11.794144 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 152/316] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:11.799771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029efda7-5c62-422a-82ce-cb53b7ac6c97 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:11.906224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029efda7-5c62-422a-82ce-cb53b7ac6c97 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:11.942547 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554c148-3219-47bd-85e6-9cd68406b9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:11.943353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554c148-3219-47bd-85e6-9cd68406b9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:11.943631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554c148-3219-47bd-85e6-9cd68406b9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:11.943904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554c148-3219-47bd-85e6-9cd68406b9d3 None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:11.945091 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554c148-3219-47bd-85e6-9cd68406b9d3 None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:11.946563 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554c148-3219-47bd-85e6-9cd68406b9d3 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:11.948455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 165/317] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:11 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:11.954574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7664d6a-3103-4215-aa62-e00ca5646362 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:12.060864 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7664d6a-3103-4215-aa62-e00ca5646362 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:12.106664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b5658-ff8f-4d30-841d-4b041d409de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:12.107338 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b5658-ff8f-4d30-841d-4b041d409de7 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:12.107632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b5658-ff8f-4d30-841d-4b041d409de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:12.107920 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b5658-ff8f-4d30-841d-4b041d409de7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:12.108766 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b5658-ff8f-4d30-841d-4b041d409de7 None admin] RBAC: Adding query filter params (role.id=6c39460198da48b7b86f53b577c698b1, scope.project.id=b271a003871a42fb99fa8b9ca8f9017f, user.id=770a31c527274941b3df3641eafdfe24) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:12.109091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b5658-ff8f-4d30-841d-4b041d409de7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:12.110688 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b5658-ff8f-4d30-841d-4b041d409de7 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:12.117529 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 153/318] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:12:11 2022] GET /identity/v3/role_assignments?user.id=770a31c527274941b3df3641eafdfe24&scope.project.id=b271a003871a42fb99fa8b9ca8f9017f&role.id=6c39460198da48b7b86f53b577c698b1 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:12.565209 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b75b0-0573-48a7-9871-570832e0e1c1 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:12.565536 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b75b0-0573-48a7-9871-570832e0e1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:12.565886 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b75b0-0573-48a7-9871-570832e0e1c1 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:12.567333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 166/319] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:12.572984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bfdfe-0515-4f41-b195-8ef365d085e7 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:12.573282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bfdfe-0515-4f41-b195-8ef365d085e7 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:12.573567 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bfdfe-0515-4f41-b195-8ef365d085e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:12.590388 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-af2bfdfe-0515-4f41-b195-8ef365d085e7 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:12.591581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-af2bfdfe-0515-4f41-b195-8ef365d085e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:12.619797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:12.661056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 154/320] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:12.684468 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b75b0-0573-48a7-9871-570832e0e1c1 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:12.713154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:12.789320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b75b0-0573-48a7-9871-570832e0e1c1 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:12.824358 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e76921-f1a7-45e6-9189-27bf06632347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:12.825086 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e76921-f1a7-45e6-9189-27bf06632347 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:12.825369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e76921-f1a7-45e6-9189-27bf06632347 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:12.825644 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e76921-f1a7-45e6-9189-27bf06632347 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:12.826859 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e76921-f1a7-45e6-9189-27bf06632347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:12.828368 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e76921-f1a7-45e6-9189-27bf06632347 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:12.830222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 167/321] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:12.835802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bfdfe-0515-4f41-b195-8ef365d085e7 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:12.906514 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f47e1a-b7a8-4b06-95ee-a9d83b89a8b2 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:12.906798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f47e1a-b7a8-4b06-95ee-a9d83b89a8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:12.907079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f47e1a-b7a8-4b06-95ee-a9d83b89a8b2 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:12.908455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 168/322] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:12:12.913948 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbc905-1cbe-4f90-aa6b-b82cfd54e4a6 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:12.914233 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbc905-1cbe-4f90-aa6b-b82cfd54e4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:12.914538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbc905-1cbe-4f90-aa6b-b82cfd54e4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:12.931582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-45dbc905-1cbe-4f90-aa6b-b82cfd54e4a6 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:12.932752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-45dbc905-1cbe-4f90-aa6b-b82cfd54e4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:12.945429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bfdfe-0515-4f41-b195-8ef365d085e7 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:12.989507 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347839ee-91ac-4210-825a-36be4dbc6cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:12.991048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347839ee-91ac-4210-825a-36be4dbc6cce None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:12.991339 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347839ee-91ac-4210-825a-36be4dbc6cce None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:12.991618 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347839ee-91ac-4210-825a-36be4dbc6cce None admin] PATH_INFO: `/v3/groups` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:12.992496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347839ee-91ac-4210-825a-36be4dbc6cce None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:12.994603 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347839ee-91ac-4210-825a-36be4dbc6cce None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:13.002725 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 169/323] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:12:13.025098 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbc905-1cbe-4f90-aa6b-b82cfd54e4a6 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:13.052034 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:13.106686 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 155/324] 10.222.0.78 () {60 vars in 1134 bytes} [Wed Jul 20 15:12:12 2022] POST /identity/v3/groups => generated 226 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:12:13.127320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbc905-1cbe-4f90-aa6b-b82cfd54e4a6 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:13.167889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cdb44-f43a-4c0d-960b-6cf80aa36d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:13.168988 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cdb44-f43a-4c0d-960b-6cf80aa36d0a None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:13.169432 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cdb44-f43a-4c0d-960b-6cf80aa36d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:13.169852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cdb44-f43a-4c0d-960b-6cf80aa36d0a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:13.181833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cdb44-f43a-4c0d-960b-6cf80aa36d0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:13.190353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-561cdb44-f43a-4c0d-960b-6cf80aa36d0a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:12:13.190353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:13.190353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:13.190353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:13.190353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:13.190353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:13.190353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:13.190353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:13.190353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:13.190353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:13.190353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:13.190353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:13.190353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:13.190353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:13.190353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:13.190353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:13.191502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:13.191502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:13.191502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:13.191502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:13.191502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:13.191502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:13.191502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:13.191502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:13.191502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:13.191502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:13.191502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:13.191502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:13.191502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:13.191502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:13.191502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:13.191502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:13.192515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:13.192515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:13.192515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:13.192515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:13.192515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:13.192515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:13.192515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:13.192515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:13.192515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:13.192515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:13.192515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:12:13.192515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:13.195028 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 170/325] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:13.200632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347839ee-91ac-4210-825a-36be4dbc6cce None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:13.229501 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:13.310760 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347839ee-91ac-4210-825a-36be4dbc6cce None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:13.346668 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ddc14-4307-4107-868f-4fe7a216e490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:13.347266 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ddc14-4307-4107-868f-4fe7a216e490 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:13.347550 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ddc14-4307-4107-868f-4fe7a216e490 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:13.347822 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ddc14-4307-4107-868f-4fe7a216e490 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:13.348563 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ddc14-4307-4107-868f-4fe7a216e490 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:13.349504 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ddc14-4307-4107-868f-4fe7a216e490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:13.350976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ddc14-4307-4107-868f-4fe7a216e490 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:13.359971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 156/326] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:12:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:13.365234 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75fe8d-4387-486c-bbec-60c62bf9ad7a None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:13.365506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75fe8d-4387-486c-bbec-60c62bf9ad7a None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:13.365738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75fe8d-4387-486c-bbec-60c62bf9ad7a None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:13.367461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 171/327] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:13.367758 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ddc14-4307-4107-868f-4fe7a216e490 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:13.372692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6ff63-cf6f-4c51-bf80-2b1febf3a001 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:13.372980 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6ff63-cf6f-4c51-bf80-2b1febf3a001 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:13.373249 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6ff63-cf6f-4c51-bf80-2b1febf3a001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:13.394466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-15e6ff63-cf6f-4c51-bf80-2b1febf3a001 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:13.399192 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-15e6ff63-cf6f-4c51-bf80-2b1febf3a001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:13.481172 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 172/328] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:13.486678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:13.505928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6ff63-cf6f-4c51-bf80-2b1febf3a001 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:13.506345 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ddc14-4307-4107-868f-4fe7a216e490 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:13.535668 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:13.550809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473985f-68fa-4aeb-b642-3524bc525051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:13.551728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473985f-68fa-4aeb-b642-3524bc525051 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:13.552046 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473985f-68fa-4aeb-b642-3524bc525051 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:13.552328 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473985f-68fa-4aeb-b642-3524bc525051 None admin] PATH_INFO: `/v3/users/8ba7526b3e9540febcdb59dc597181a8` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:13.554074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473985f-68fa-4aeb-b642-3524bc525051 None admin] RBAC: Authorizing `identity:get_user(user_id=8ba7526b3e9540febcdb59dc597181a8)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:13.556671 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473985f-68fa-4aeb-b642-3524bc525051 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:13.559216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 157/329] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:12:13 2022] GET /identity/v3/users/8ba7526b3e9540febcdb59dc597181a8 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:13.564238 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473985f-68fa-4aeb-b642-3524bc525051 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:13.648335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6ff63-cf6f-4c51-bf80-2b1febf3a001 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:13.681486 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473985f-68fa-4aeb-b642-3524bc525051 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:13.689217 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0ab8c-1271-4cc1-90a5-3ddb08a82f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:13.690024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0ab8c-1271-4cc1-90a5-3ddb08a82f24 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:13.690338 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0ab8c-1271-4cc1-90a5-3ddb08a82f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:13.690615 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0ab8c-1271-4cc1-90a5-3ddb08a82f24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:13.703485 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0ab8c-1271-4cc1-90a5-3ddb08a82f24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:13.713230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-45b0ab8c-1271-4cc1-90a5-3ddb08a82f24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:13.713230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:13.713230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:13.713230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:13.713230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:13.713230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:13.713230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:13.713230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:13.713230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:13.713230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:13.713230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:13.713230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:13.713230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:13.713230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:13.713230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:13.713230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:13.714204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:13.714204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:13.714204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:13.714204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:13.714204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:13.714204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:13.714204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:13.714204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:13.714204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:13.714204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:13.714204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:13.714204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:13.714204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:13.714204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:13.714204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:13.714204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:13.715050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:13.715050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:13.715050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:13.715050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:13.715050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:13.715050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:13.715050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:13.715050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:13.715050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:13.715050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:13.715050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:13.715050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:13.715050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 173/330] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:13.722859 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0ab8c-1271-4cc1-90a5-3ddb08a82f24 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:13.730565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964c3bb-33a1-4940-bd07-22d8da1bc243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:13.731749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964c3bb-33a1-4940-bd07-22d8da1bc243 None admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:13.732233 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964c3bb-33a1-4940-bd07-22d8da1bc243 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:13.732709 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964c3bb-33a1-4940-bd07-22d8da1bc243 None admin] PATH_INFO: `/v3/system/users/8ba7526b3e9540febcdb59dc597181a8/roles/6c39460198da48b7b86f53b577c698b1` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:13.734516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964c3bb-33a1-4940-bd07-22d8da1bc243 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8ba7526b3e9540febcdb59dc597181a8, role_id=6c39460198da48b7b86f53b577c698b1)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:13.736161 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964c3bb-33a1-4940-bd07-22d8da1bc243 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:13.811704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:13.835476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0ab8c-1271-4cc1-90a5-3ddb08a82f24 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:13.841771 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 158/331] 10.222.0.78 () {58 vars in 1334 bytes} [Wed Jul 20 15:12:13 2022] PUT /identity/v3/system/users/8ba7526b3e9540febcdb59dc597181a8/roles/6c39460198da48b7b86f53b577c698b1 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:12:13.879205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4adb40-4e6e-4cf9-bb45-2a955297657f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:13.881125 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4adb40-4e6e-4cf9-bb45-2a955297657f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:13.881595 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4adb40-4e6e-4cf9-bb45-2a955297657f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:13.882098 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4adb40-4e6e-4cf9-bb45-2a955297657f None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:13.883134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4adb40-4e6e-4cf9-bb45-2a955297657f None admin] RBAC: Adding query filter params (name=member) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:13.883635 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4adb40-4e6e-4cf9-bb45-2a955297657f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:13.885374 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4adb40-4e6e-4cf9-bb45-2a955297657f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:13.896902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 174/332] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:12:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:13.903297 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964c3bb-33a1-4940-bd07-22d8da1bc243 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:13.928905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:14.029181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964c3bb-33a1-4940-bd07-22d8da1bc243 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:14.065270 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1c250-fbcf-482f-b8ac-d474344d8f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:14.066200 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1c250-fbcf-482f-b8ac-d474344d8f41 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:14.066478 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1c250-fbcf-482f-b8ac-d474344d8f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:14.066759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1c250-fbcf-482f-b8ac-d474344d8f41 None admin] PATH_INFO: `/v3/users/c572a3809e76418a9879c801a3feeb13` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:14.068566 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1c250-fbcf-482f-b8ac-d474344d8f41 None admin] RBAC: Authorizing `identity:get_user(user_id=c572a3809e76418a9879c801a3feeb13)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:14.070017 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1c250-fbcf-482f-b8ac-d474344d8f41 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:14.072090 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 159/333] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:12:13 2022] GET /identity/v3/users/c572a3809e76418a9879c801a3feeb13 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:14.077888 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4adb40-4e6e-4cf9-bb45-2a955297657f None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:14.188491 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4adb40-4e6e-4cf9-bb45-2a955297657f None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:14.228858 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3eae54-5773-4a79-8351-f61800d44d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:14.229765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eae54-5773-4a79-8351-f61800d44d0c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:14.230196 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eae54-5773-4a79-8351-f61800d44d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:14.230656 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eae54-5773-4a79-8351-f61800d44d0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:14.231638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3eae54-5773-4a79-8351-f61800d44d0c None admin] RBAC: Adding query filter params (role.id=d85783606f6649f5b9465f407cafb610, user.id=c572a3809e76418a9879c801a3feeb13, scope.system=all) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:14.232112 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3eae54-5773-4a79-8351-f61800d44d0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:14.233710 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3eae54-5773-4a79-8351-f61800d44d0c None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:14.240075 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:14.253040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 175/334] 10.222.0.78 () {56 vars in 1308 bytes} [Wed Jul 20 15:12:14 2022] GET /identity/v3/role_assignments?user.id=c572a3809e76418a9879c801a3feeb13&scope.system=all&role.id=d85783606f6649f5b9465f407cafb610 => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:16.141809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b82e8-b405-4c7b-826d-7e23e1b82fd1 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:16.142412 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b82e8-b405-4c7b-826d-7e23e1b82fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:16.142412 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b82e8-b405-4c7b-826d-7e23e1b82fd1 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:16.144117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 160/335] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:16.150074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec88801-89db-46d7-ab3c-784518d2e879 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:16.151063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec88801-89db-46d7-ab3c-784518d2e879 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:16.151550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec88801-89db-46d7-ab3c-784518d2e879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:16.179418 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-fec88801-89db-46d7-ab3c-784518d2e879 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:16.180982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-fec88801-89db-46d7-ab3c-784518d2e879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:16.232965 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b87553-f0bc-45e4-9f1a-1204a8acab5d None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:16.233314 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b87553-f0bc-45e4-9f1a-1204a8acab5d None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:16.233528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b87553-f0bc-45e4-9f1a-1204a8acab5d None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:16.236584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 161/336] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:16.241438 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fcec2-f7ad-414b-a38b-c4356c7bfe7d None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:16.241728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fcec2-f7ad-414b-a38b-c4356c7bfe7d None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:16.242007 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fcec2-f7ad-414b-a38b-c4356c7bfe7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:16.274356 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-315fcec2-f7ad-414b-a38b-c4356c7bfe7d None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:16.274356 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-315fcec2-f7ad-414b-a38b-c4356c7bfe7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:16.281655 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 176/337] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:16.313682 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec88801-89db-46d7-ab3c-784518d2e879 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:16.424283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 162/338] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:16.425551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265cce2-6baf-4b23-822c-e3ea59485bdd None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:16.425837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265cce2-6baf-4b23-822c-e3ea59485bdd None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:16.426113 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265cce2-6baf-4b23-822c-e3ea59485bdd None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:16.428115 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 163/339] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:16.433351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29152196-4fda-479a-ac54-c2f6c31da0b8 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:16.433637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29152196-4fda-479a-ac54-c2f6c31da0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:16.433917 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29152196-4fda-479a-ac54-c2f6c31da0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:16.438514 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec88801-89db-46d7-ab3c-784518d2e879 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:16.459885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-29152196-4fda-479a-ac54-c2f6c31da0b8 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:16.461071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-29152196-4fda-479a-ac54-c2f6c31da0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:16.471409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:16.480288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:16.484683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeac6b0-8a70-4006-9028-0698f029d513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:16.485915 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeac6b0-8a70-4006-9028-0698f029d513 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:16.486448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeac6b0-8a70-4006-9028-0698f029d513 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:16.486938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeac6b0-8a70-4006-9028-0698f029d513 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:16.499307 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeac6b0-8a70-4006-9028-0698f029d513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:16.508584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-efeac6b0-8a70-4006-9028-0698f029d513 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:16.508584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:16.508584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:16.508584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:16.508584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:16.508584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:16.508584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:16.508584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:16.508584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:16.508584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:16.508584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:16.508584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:16.508584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:16.508584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:16.508584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:16.508584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:16.509589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:16.509589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:16.509589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:16.509589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:16.509589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:16.509589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:16.509589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:16.509589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:16.509589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:16.509589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:16.509589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:16.509589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:16.509589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:16.509589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:16.509589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:16.509589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:16.510476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:16.510476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:16.510476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:16.510476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:16.510476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:16.510476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:16.510476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:16.510476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:16.510476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:16.510476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:16.510476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:16.510476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:16.513195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 177/340] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:12:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:16.514286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeac6b0-8a70-4006-9028-0698f029d513 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:16.568860 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 164/341] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:16.569748 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29152196-4fda-479a-ac54-c2f6c31da0b8 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:16.624515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeac6b0-8a70-4006-9028-0698f029d513 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:16.667149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db3435-ab9c-4c0c-ac9b-45a6cade9db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:16.667964 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db3435-ab9c-4c0c-ac9b-45a6cade9db9 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:16.668244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db3435-ab9c-4c0c-ac9b-45a6cade9db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:16.668516 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db3435-ab9c-4c0c-ac9b-45a6cade9db9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:16.679984 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db3435-ab9c-4c0c-ac9b-45a6cade9db9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:16.688685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-b0db3435-ab9c-4c0c-ac9b-45a6cade9db9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:16.688685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:16.688685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:16.688685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:16.688685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:16.688685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:16.688685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:16.688685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:16.688685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:16.688685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:16.688685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:16.688685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:16.688685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:16.688685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:16.688685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:16.688685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:16.689592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:16.689592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:16.689592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:16.689592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:16.689592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:16.689592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:16.689592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:16.689592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:16.689592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:16.689592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:16.689592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:16.689592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:16.689592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:16.689592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:16.689592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:16.689592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:16.690440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:16.690440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:16.690440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:16.690440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:16.690440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:16.690440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:16.690440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:16.690440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:16.690440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:16.690440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:16.690440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:16.690440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:16.690440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 178/342] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:16.691132 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db3435-ab9c-4c0c-ac9b-45a6cade9db9 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:16.709491 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29152196-4fda-479a-ac54-c2f6c31da0b8 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:16.756434 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d512f4b-b62b-45fe-9c35-5bc910f54b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:16.757694 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:16.759178 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d512f4b-b62b-45fe-9c35-5bc910f54b7f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:16.759728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d512f4b-b62b-45fe-9c35-5bc910f54b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:16.760159 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d512f4b-b62b-45fe-9c35-5bc910f54b7f None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:16.761239 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d512f4b-b62b-45fe-9c35-5bc910f54b7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:16.762042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d512f4b-b62b-45fe-9c35-5bc910f54b7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:16.763630 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d512f4b-b62b-45fe-9c35-5bc910f54b7f None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:16.775837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 165/343] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:16.778861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d512f4b-b62b-45fe-9c35-5bc910f54b7f None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:16.790746 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:16.839691 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db3435-ab9c-4c0c-ac9b-45a6cade9db9 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:16.892541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2a9cd-0ea2-42b5-b6e3-593e0fcc40a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:16.893655 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2a9cd-0ea2-42b5-b6e3-593e0fcc40a6 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:16.894090 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2a9cd-0ea2-42b5-b6e3-593e0fcc40a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:16.894391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2a9cd-0ea2-42b5-b6e3-593e0fcc40a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:16.908883 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc2a9cd-0ea2-42b5-b6e3-593e0fcc40a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:16.916843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-ecc2a9cd-0ea2-42b5-b6e3-593e0fcc40a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:12:16.916843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:16.916843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:16.916843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:16.916843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:16.916843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:16.916843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:16.916843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:16.916843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:16.916843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:16.916843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:16.916843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:16.916843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:16.916843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:16.916843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:16.916843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:16.917804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:16.917804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:16.917804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:16.917804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:16.917804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:16.917804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:16.917804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:16.917804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:16.917804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:16.917804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:16.917804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:16.917804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:16.917804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:16.917804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:16.917804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:16.917804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:16.919195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:16.919195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:16.919195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:16.919195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:16.919195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:16.919195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:16.919195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:16.919195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:16.919195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:16.919195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:16.919195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 15:12:16.919195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:16.919195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 179/344] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:16.920208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2a9cd-0ea2-42b5-b6e3-593e0fcc40a6 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:16.972921 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d512f4b-b62b-45fe-9c35-5bc910f54b7f None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:17.013282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252af90b-be63-4bc4-8a82-74b5afc37b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:17.014080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252af90b-be63-4bc4-8a82-74b5afc37b19 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:17.014431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252af90b-be63-4bc4-8a82-74b5afc37b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:17.014793 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252af90b-be63-4bc4-8a82-74b5afc37b19 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:17.015550 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252af90b-be63-4bc4-8a82-74b5afc37b19 None admin] RBAC: Adding query filter params (name=member) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:17.015859 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252af90b-be63-4bc4-8a82-74b5afc37b19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:17.017278 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252af90b-be63-4bc4-8a82-74b5afc37b19 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:17.030641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 166/345] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:12:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:17.031285 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252af90b-be63-4bc4-8a82-74b5afc37b19 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:17.058511 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2a9cd-0ea2-42b5-b6e3-593e0fcc40a6 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:17.066215 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:17.099195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9b446-f4e8-4a74-884f-c6fdabcebfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:17.099966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9b446-f4e8-4a74-884f-c6fdabcebfcf None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:17.100255 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9b446-f4e8-4a74-884f-c6fdabcebfcf None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:17.100532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9b446-f4e8-4a74-884f-c6fdabcebfcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:17.122019 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:17.165862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252af90b-be63-4bc4-8a82-74b5afc37b19 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:17.190183 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9b446-f4e8-4a74-884f-c6fdabcebfcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:17.191816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9b446-f4e8-4a74-884f-c6fdabcebfcf None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:17.210056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fea18-9752-4479-97b7-31200b36a846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:17.210981 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fea18-9752-4479-97b7-31200b36a846 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:17.211663 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fea18-9752-4479-97b7-31200b36a846 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:17.211945 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fea18-9752-4479-97b7-31200b36a846 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:17.212696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fea18-9752-4479-97b7-31200b36a846 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:17.213002 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fea18-9752-4479-97b7-31200b36a846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:17.214874 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fea18-9752-4479-97b7-31200b36a846 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:17.226981 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 167/346] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:12:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:17.227981 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a796c8-f3e9-4581-9e9e-1db54179d611 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:17.228273 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a796c8-f3e9-4581-9e9e-1db54179d611 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:17.228557 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a796c8-f3e9-4581-9e9e-1db54179d611 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:17.234796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 168/347] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:17.235627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a796c8-f3e9-4581-9e9e-1db54179d611 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:17.325652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 180/348] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:17.327036 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d76f45-6b79-45d0-a2fa-195908c1cfa6 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:17.327391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d76f45-6b79-45d0-a2fa-195908c1cfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:17.327671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d76f45-6b79-45d0-a2fa-195908c1cfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:17.344993 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:17.354134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-90d76f45-6b79-45d0-a2fa-195908c1cfa6 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:17.361958 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-90d76f45-6b79-45d0-a2fa-195908c1cfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:17.374340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a796c8-f3e9-4581-9e9e-1db54179d611 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:17.429020 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:17.438974 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b69a2c-6310-45cd-839e-90a1dce3a1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:17.442327 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 181/349] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:17.442509 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b69a2c-6310-45cd-839e-90a1dce3a1cd None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:17.442654 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d76f45-6b79-45d0-a2fa-195908c1cfa6 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:17.442776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b69a2c-6310-45cd-839e-90a1dce3a1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:17.442776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b69a2c-6310-45cd-839e-90a1dce3a1cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:17.522710 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b69a2c-6310-45cd-839e-90a1dce3a1cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:17.524263 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b69a2c-6310-45cd-839e-90a1dce3a1cd None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:17.572410 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d76f45-6b79-45d0-a2fa-195908c1cfa6 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:17.617772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7324d-d809-4733-956d-ed7ec9ca0b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:17.619142 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7324d-d809-4733-956d-ed7ec9ca0b25 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:17.619595 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7324d-d809-4733-956d-ed7ec9ca0b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:17.619873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7324d-d809-4733-956d-ed7ec9ca0b25 None admin] PATH_INFO: `/v3/users/8ba7526b3e9540febcdb59dc597181a8` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:17.622680 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7324d-d809-4733-956d-ed7ec9ca0b25 None admin] RBAC: Authorizing `identity:get_user(user_id=8ba7526b3e9540febcdb59dc597181a8)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:17.625562 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7324d-d809-4733-956d-ed7ec9ca0b25 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:17.628163 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 182/350] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:12:17 2022] GET /identity/v3/users/8ba7526b3e9540febcdb59dc597181a8 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:17.629118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7324d-d809-4733-956d-ed7ec9ca0b25 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:17.633710 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:17.650650 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 169/351] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:17.652699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d89d7-6397-4ac4-95a3-536319bfe472 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:17.653146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d89d7-6397-4ac4-95a3-536319bfe472 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:17.653628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d89d7-6397-4ac4-95a3-536319bfe472 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:17.656440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 170/352] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:17.657477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d89d7-6397-4ac4-95a3-536319bfe472 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:17.696810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:17.755751 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7324d-d809-4733-956d-ed7ec9ca0b25 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:17.798682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d89d7-6397-4ac4-95a3-536319bfe472 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:17.805680 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03917463-6bdb-4e69-a4eb-f4eb13815736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:17.806878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03917463-6bdb-4e69-a4eb-f4eb13815736 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:17.807384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03917463-6bdb-4e69-a4eb-f4eb13815736 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:17.807878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03917463-6bdb-4e69-a4eb-f4eb13815736 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:17.809786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03917463-6bdb-4e69-a4eb-f4eb13815736 None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:17.812287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03917463-6bdb-4e69-a4eb-f4eb13815736 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:17.815043 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 183/353] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:17 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:17.816443 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03917463-6bdb-4e69-a4eb-f4eb13815736 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:17.842704 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9def7-c115-4f10-877a-ada94ee05db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:17.843511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9def7-c115-4f10-877a-ada94ee05db7 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:17.843795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9def7-c115-4f10-877a-ada94ee05db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:17.844073 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9def7-c115-4f10-877a-ada94ee05db7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:17.857232 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9def7-c115-4f10-877a-ada94ee05db7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:17.865616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-caf9def7-c115-4f10-877a-ada94ee05db7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:17.865616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:17.865616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:17.865616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:17.865616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:17.865616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:17.865616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:17.865616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:17.865616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:17.865616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:17.865616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:17.865616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:17.865616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:17.865616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:17.865616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:17.865616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:17.866574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:17.866574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:17.866574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:17.866574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:17.866574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:17.866574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:17.866574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:17.866574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:17.866574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:17.866574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:17.866574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:17.866574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:17.866574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:17.866574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:17.866574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:17.866574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:17.867394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:17.867394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:17.867394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:17.867394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:17.867394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:17.867394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:17.867394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:17.867394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:17.867394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:17.867394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:17.867394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:17.867394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:17.867394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 171/354] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:17.868056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9def7-c115-4f10-877a-ada94ee05db7 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:17.908821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:17.948319 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03917463-6bdb-4e69-a4eb-f4eb13815736 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:17.985634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:17.986438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000907b1-df6d-4248-b331-8b4d6c2e2075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:17.987318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000907b1-df6d-4248-b331-8b4d6c2e2075 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:17.987631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000907b1-df6d-4248-b331-8b4d6c2e2075 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:17.987913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000907b1-df6d-4248-b331-8b4d6c2e2075 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:17.988762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000907b1-df6d-4248-b331-8b4d6c2e2075 None admin] RBAC: Adding query filter params (role.id=6c39460198da48b7b86f53b577c698b1, user.id=8ba7526b3e9540febcdb59dc597181a8, scope.system=all) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:17.989067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000907b1-df6d-4248-b331-8b4d6c2e2075 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:17.990597 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000907b1-df6d-4248-b331-8b4d6c2e2075 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:17.995748 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9def7-c115-4f10-877a-ada94ee05db7 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:18.005468 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 184/355] 10.222.0.78 () {56 vars in 1308 bytes} [Wed Jul 20 15:12:17 2022] GET /identity/v3/role_assignments?user.id=8ba7526b3e9540febcdb59dc597181a8&scope.system=all&role.id=6c39460198da48b7b86f53b577c698b1 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:18.006880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f6b4a-2810-4784-a0b0-c561c75a60c4 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:18.007287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f6b4a-2810-4784-a0b0-c561c75a60c4 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:18.008741 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f6b4a-2810-4784-a0b0-c561c75a60c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:18.027884 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-f83f6b4a-2810-4784-a0b0-c561c75a60c4 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:18.029115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f83f6b4a-2810-4784-a0b0-c561c75a60c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:18.049082 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f6662-4d91-4372-87d4-d5f8b90eb589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:18.050885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f6662-4d91-4372-87d4-d5f8b90eb589 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:18.051180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f6662-4d91-4372-87d4-d5f8b90eb589 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:18.051466 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f6662-4d91-4372-87d4-d5f8b90eb589 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:18.053396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f6662-4d91-4372-87d4-d5f8b90eb589 None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:18.054960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f6662-4d91-4372-87d4-d5f8b90eb589 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:18.056747 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 172/356] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:17 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:18.061394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f6662-4d91-4372-87d4-d5f8b90eb589 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:18.124652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 185/357] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:18.125875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f6b4a-2810-4784-a0b0-c561c75a60c4 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:18.199346 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f6662-4d91-4372-87d4-d5f8b90eb589 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:18.200373 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:18.249851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c0fcc-e3fc-4ee3-abc0-b50eecbe4bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:18.250450 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f6b4a-2810-4784-a0b0-c561c75a60c4 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:18.250959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c0fcc-e3fc-4ee3-abc0-b50eecbe4bba None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:18.251247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c0fcc-e3fc-4ee3-abc0-b50eecbe4bba None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:18.251530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c0fcc-e3fc-4ee3-abc0-b50eecbe4bba None admin] PATH_INFO: `/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:18.259160 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c0fcc-e3fc-4ee3-abc0-b50eecbe4bba None admin] RBAC: Authorizing `identity:get_group(group_id=921a4e3d2fdf49ba82cc42dc6f7f9737)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:18.260548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c0fcc-e3fc-4ee3-abc0-b50eecbe4bba None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:18.262242 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 173/358] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Jul 20 15:12:18 2022] GET /identity/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:18.264024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828da50d-2604-436e-9d3c-3cc283f3ba4f None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:18.264311 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828da50d-2604-436e-9d3c-3cc283f3ba4f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:18.264591 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828da50d-2604-436e-9d3c-3cc283f3ba4f None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:18.267067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 174/359] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:18.268295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828da50d-2604-436e-9d3c-3cc283f3ba4f None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:18.289041 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb29048-99f6-4640-8230-0a8e6f176c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:18.289975 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb29048-99f6-4640-8230-0a8e6f176c29 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:18.290455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb29048-99f6-4640-8230-0a8e6f176c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:18.290880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb29048-99f6-4640-8230-0a8e6f176c29 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:18.291811 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb29048-99f6-4640-8230-0a8e6f176c29 None admin] RBAC: Adding query filter params (name=member) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:18.292123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb29048-99f6-4640-8230-0a8e6f176c29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:18.293514 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb29048-99f6-4640-8230-0a8e6f176c29 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:18.293896 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:18.310740 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 186/360] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:12:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:18.310740 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb29048-99f6-4640-8230-0a8e6f176c29 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:18.393567 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828da50d-2604-436e-9d3c-3cc283f3ba4f None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:18.423114 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb29048-99f6-4640-8230-0a8e6f176c29 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:18.433501 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b3237-09f7-4822-8f2d-759732d19538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:18.434411 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b3237-09f7-4822-8f2d-759732d19538 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:18.434693 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b3237-09f7-4822-8f2d-759732d19538 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:18.434964 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b3237-09f7-4822-8f2d-759732d19538 None admin] PATH_INFO: `/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:18.436932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b3237-09f7-4822-8f2d-759732d19538 None admin] RBAC: Authorizing `identity:get_group(group_id=921a4e3d2fdf49ba82cc42dc6f7f9737)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:18.438549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b3237-09f7-4822-8f2d-759732d19538 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:18.440167 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 175/361] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Jul 20 15:12:18 2022] GET /identity/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:18.441014 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b3237-09f7-4822-8f2d-759732d19538 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:18.451270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:18.468385 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140952b2-65ab-43e4-b413-d3daf63bc86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:18.469332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140952b2-65ab-43e4-b413-d3daf63bc86b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:18.469625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140952b2-65ab-43e4-b413-d3daf63bc86b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:18.475798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140952b2-65ab-43e4-b413-d3daf63bc86b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:18.480737 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140952b2-65ab-43e4-b413-d3daf63bc86b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:18.488871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-140952b2-65ab-43e4-b413-d3daf63bc86b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:18.488871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:18.488871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:18.488871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:18.488871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:18.488871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:18.488871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:18.488871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:18.488871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:18.488871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:18.488871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:18.488871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:18.488871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:18.488871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:18.488871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:18.488871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:18.490477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:18.490477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:18.490477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:18.490477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:18.490477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:18.490477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:18.490477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:18.490477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:18.490477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:18.490477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:18.490477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:18.490477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:18.490477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:18.490477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:18.490477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:18.490477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:18.491310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:18.491310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:18.491310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:18.491310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:18.491310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:18.491310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:18.491310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:18.491310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:18.491310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:18.491310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:18.491310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:18.491310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:18.491310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 187/362] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:12:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:18.492016 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fcaa9-7b05-4926-9533-0bf7bf346cb6 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:18.492609 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fcaa9-7b05-4926-9533-0bf7bf346cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:18.492609 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fcaa9-7b05-4926-9533-0bf7bf346cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:18.511194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6a1fcaa9-7b05-4926-9533-0bf7bf346cb6 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:18.512393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6a1fcaa9-7b05-4926-9533-0bf7bf346cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:18.546497 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:18.557838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b3237-09f7-4822-8f2d-759732d19538 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:18.594979 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 188/363] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:18.596101 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fcaa9-7b05-4926-9533-0bf7bf346cb6 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:18.600287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdefc7d-2810-4731-a4ef-d095c81d42bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:18.600994 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdefc7d-2810-4731-a4ef-d095c81d42bf None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:18.601307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdefc7d-2810-4731-a4ef-d095c81d42bf None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:18.601644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdefc7d-2810-4731-a4ef-d095c81d42bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:18.603187 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdefc7d-2810-4731-a4ef-d095c81d42bf None admin] RBAC: Adding query filter params (group.id=921a4e3d2fdf49ba82cc42dc6f7f9737, role.id=2da8d544efe44ce6a41160d5759be402, scope.project.id=ba23663aa02d442ab621813797163d82) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:18.603187 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdefc7d-2810-4731-a4ef-d095c81d42bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:18.604847 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdefc7d-2810-4731-a4ef-d095c81d42bf None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:18.611728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 176/364] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Jul 20 15:12:18 2022] GET /identity/v3/role_assignments?group.id=921a4e3d2fdf49ba82cc42dc6f7f9737&scope.project.id=ba23663aa02d442ab621813797163d82&role.id=2da8d544efe44ce6a41160d5759be402 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:18.613430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdefc7d-2810-4731-a4ef-d095c81d42bf None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:18.731765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fcaa9-7b05-4926-9533-0bf7bf346cb6 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:18.749197 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:18.761910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdefc7d-2810-4731-a4ef-d095c81d42bf None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:18.785939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922af5a4-b544-413d-9883-bbfa7767a641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:18.787065 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922af5a4-b544-413d-9883-bbfa7767a641 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:18.787568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922af5a4-b544-413d-9883-bbfa7767a641 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:18.788146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922af5a4-b544-413d-9883-bbfa7767a641 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:18.815004 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d4b37-8c62-4992-8314-cfd615917dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:18.816063 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d4b37-8c62-4992-8314-cfd615917dcf None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:18.816846 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d4b37-8c62-4992-8314-cfd615917dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:18.817136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d4b37-8c62-4992-8314-cfd615917dcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:18.818584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d4b37-8c62-4992-8314-cfd615917dcf None admin] RBAC: Adding query filter params (group.id=921a4e3d2fdf49ba82cc42dc6f7f9737, role.id=d85783606f6649f5b9465f407cafb610, scope.project.id=ba23663aa02d442ab621813797163d82) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:18.818937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d4b37-8c62-4992-8314-cfd615917dcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:18.820361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d4b37-8c62-4992-8314-cfd615917dcf None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:18.826581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 177/365] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Jul 20 15:12:18 2022] GET /identity/v3/role_assignments?group.id=921a4e3d2fdf49ba82cc42dc6f7f9737&scope.project.id=ba23663aa02d442ab621813797163d82&role.id=d85783606f6649f5b9465f407cafb610 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:18.828981 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d4b37-8c62-4992-8314-cfd615917dcf None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:18.864008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:18.880765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922af5a4-b544-413d-9883-bbfa7767a641 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:18.882600 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922af5a4-b544-413d-9883-bbfa7767a641 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:18.968675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d4b37-8c62-4992-8314-cfd615917dcf None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:19.010611 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 189/366] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:19.016151 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922af5a4-b544-413d-9883-bbfa7767a641 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:19.024887 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f14d44-b33a-4972-b558-39b4a51dbb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:19.025679 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f14d44-b33a-4972-b558-39b4a51dbb68 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:19.026014 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f14d44-b33a-4972-b558-39b4a51dbb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:19.026384 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f14d44-b33a-4972-b558-39b4a51dbb68 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:19.027243 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f14d44-b33a-4972-b558-39b4a51dbb68 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:19.027625 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f14d44-b33a-4972-b558-39b4a51dbb68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:19.029927 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f14d44-b33a-4972-b558-39b4a51dbb68 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:19.035330 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:19.046415 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 178/367] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:19.047706 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f14d44-b33a-4972-b558-39b4a51dbb68 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:19.165253 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:19.178376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922af5a4-b544-413d-9883-bbfa7767a641 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:19.187773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f14d44-b33a-4972-b558-39b4a51dbb68 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:19.218985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b599fb1-987d-4e4c-8994-5f9ce8e9e00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:19.220547 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b599fb1-987d-4e4c-8994-5f9ce8e9e00d None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:19.220921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b599fb1-987d-4e4c-8994-5f9ce8e9e00d None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:19.221440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b599fb1-987d-4e4c-8994-5f9ce8e9e00d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:19.231259 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954fc45-fce6-422f-a252-9b83e97c3111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:19.232419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954fc45-fce6-422f-a252-9b83e97c3111 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:19.232702 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954fc45-fce6-422f-a252-9b83e97c3111 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:19.232983 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954fc45-fce6-422f-a252-9b83e97c3111 None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:19.234499 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954fc45-fce6-422f-a252-9b83e97c3111 None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:19.234608 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b599fb1-987d-4e4c-8994-5f9ce8e9e00d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:19.235936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954fc45-fce6-422f-a252-9b83e97c3111 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:19.237639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 179/368] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:19 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:19.238435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954fc45-fce6-422f-a252-9b83e97c3111 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:19.248687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-1b599fb1-987d-4e4c-8994-5f9ce8e9e00d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:12:19.248687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:19.248687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:19.248687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:19.248687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:19.248687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:19.248687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:19.248687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:19.248687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:19.248687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:19.248687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:19.248687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:19.248687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:19.248687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:19.248687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:19.248687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:19.250868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:19.250868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:19.250868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:19.250868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:19.250868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:19.250868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:19.250868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:19.250868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:19.250868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:19.250868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:19.250868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:19.250868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:19.250868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:19.250868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:19.250868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:19.250868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:19.252924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:19.252924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:19.252924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:19.252924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:19.252924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:19.252924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:19.252924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:19.252924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:19.252924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:19.252924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:19.252924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:12:19.252924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:19.252924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 190/369] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:12:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:19.252924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b599fb1-987d-4e4c-8994-5f9ce8e9e00d None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:19.353663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:19.374859 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954fc45-fce6-422f-a252-9b83e97c3111 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:19.396053 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b599fb1-987d-4e4c-8994-5f9ce8e9e00d None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:19.416846 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5f08f-fbef-40bb-8b96-811ac7dd96ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:19.417425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5f08f-fbef-40bb-8b96-811ac7dd96ff None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:19.417709 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5f08f-fbef-40bb-8b96-811ac7dd96ff None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:19.417994 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5f08f-fbef-40bb-8b96-811ac7dd96ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:19.438620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:19.447021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01032063-3d75-47d6-a9a8-775fe35b81d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:19.448996 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01032063-3d75-47d6-a9a8-775fe35b81d5 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:19.449298 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01032063-3d75-47d6-a9a8-775fe35b81d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:19.451520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01032063-3d75-47d6-a9a8-775fe35b81d5 None admin] PATH_INFO: `/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:19.451685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01032063-3d75-47d6-a9a8-775fe35b81d5 None admin] RBAC: Authorizing `identity:get_group(group_id=921a4e3d2fdf49ba82cc42dc6f7f9737)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:19.453093 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01032063-3d75-47d6-a9a8-775fe35b81d5 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:19.455090 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 191/370] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Jul 20 15:12:19 2022] GET /identity/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:19.455862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01032063-3d75-47d6-a9a8-775fe35b81d5 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:19.517169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a5f08f-fbef-40bb-8b96-811ac7dd96ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:19.519535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a5f08f-fbef-40bb-8b96-811ac7dd96ff None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:19.581447 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01032063-3d75-47d6-a9a8-775fe35b81d5 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:19.628418 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99e36a-e8a3-4b87-8770-894272faa41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:19.629391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99e36a-e8a3-4b87-8770-894272faa41f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:19.629672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99e36a-e8a3-4b87-8770-894272faa41f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:19.630502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99e36a-e8a3-4b87-8770-894272faa41f None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:19.631406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99e36a-e8a3-4b87-8770-894272faa41f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:19.631802 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99e36a-e8a3-4b87-8770-894272faa41f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:19.633778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99e36a-e8a3-4b87-8770-894272faa41f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:19.639676 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 180/371] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:19.640773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5f08f-fbef-40bb-8b96-811ac7dd96ff None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:19.649667 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 192/372] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:19.650530 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99e36a-e8a3-4b87-8770-894272faa41f None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:19.655156 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:19.722835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:19.775073 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99e36a-e8a3-4b87-8770-894272faa41f None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:19.778335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5f08f-fbef-40bb-8b96-811ac7dd96ff None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:19.811548 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ebd22-013c-45ac-b12a-5945a881548f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:19.812764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ebd22-013c-45ac-b12a-5945a881548f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:19.813268 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ebd22-013c-45ac-b12a-5945a881548f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:19.813741 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ebd22-013c-45ac-b12a-5945a881548f None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:19.815334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ebd22-013c-45ac-b12a-5945a881548f None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:19.817191 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ebd22-013c-45ac-b12a-5945a881548f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:19.818856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac74ef-a6d7-4858-8875-c78cf54ea8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:19.819574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac74ef-a6d7-4858-8875-c78cf54ea8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:19.819855 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac74ef-a6d7-4858-8875-c78cf54ea8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:19.820128 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac74ef-a6d7-4858-8875-c78cf54ea8f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:19.820888 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 193/373] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:19 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:19.821967 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ebd22-013c-45ac-b12a-5945a881548f None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:19.825085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac74ef-a6d7-4858-8875-c78cf54ea8f5 None admin] RBAC: Adding query filter params (group.id=921a4e3d2fdf49ba82cc42dc6f7f9737, role.id=d85783606f6649f5b9465f407cafb610, scope.project.id=b271a003871a42fb99fa8b9ca8f9017f) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:19.825436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac74ef-a6d7-4858-8875-c78cf54ea8f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:19.827243 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac74ef-a6d7-4858-8875-c78cf54ea8f5 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:19.836021 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 181/374] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Jul 20 15:12:19 2022] GET /identity/v3/role_assignments?group.id=921a4e3d2fdf49ba82cc42dc6f7f9737&scope.project.id=b271a003871a42fb99fa8b9ca8f9017f&role.id=d85783606f6649f5b9465f407cafb610 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:19.837411 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac74ef-a6d7-4858-8875-c78cf54ea8f5 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:19.929732 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:19.955319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac74ef-a6d7-4858-8875-c78cf54ea8f5 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:19.973784 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:19.975244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ebd22-013c-45ac-b12a-5945a881548f None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:19.995160 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb76c2-7083-41b3-a173-1821034275a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:19.996193 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb76c2-7083-41b3-a173-1821034275a9 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:19.996518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb76c2-7083-41b3-a173-1821034275a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:19.996809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb76c2-7083-41b3-a173-1821034275a9 None admin] PATH_INFO: `/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:19.999551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb76c2-7083-41b3-a173-1821034275a9 None admin] RBAC: Authorizing `identity:get_group(group_id=921a4e3d2fdf49ba82cc42dc6f7f9737)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:20.001031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb76c2-7083-41b3-a173-1821034275a9 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:20.008967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 182/375] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Jul 20 15:12:19 2022] GET /identity/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:20.010401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb76c2-7083-41b3-a173-1821034275a9 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:20.046222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38524e60-0030-4652-ab67-3d98687d1569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:20.047020 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38524e60-0030-4652-ab67-3d98687d1569 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:20.047309 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38524e60-0030-4652-ab67-3d98687d1569 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:20.047594 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38524e60-0030-4652-ab67-3d98687d1569 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:20.127101 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb76c2-7083-41b3-a173-1821034275a9 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:20.166365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f561fe-95f9-4c36-b3b9-685ade2a2724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:20.167028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f561fe-95f9-4c36-b3b9-685ade2a2724 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:20.167312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f561fe-95f9-4c36-b3b9-685ade2a2724 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:20.167588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f561fe-95f9-4c36-b3b9-685ade2a2724 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:20.168417 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f561fe-95f9-4c36-b3b9-685ade2a2724 None admin] RBAC: Adding query filter params (group.id=921a4e3d2fdf49ba82cc42dc6f7f9737, role.id=2da8d544efe44ce6a41160d5759be402, scope.project.id=b271a003871a42fb99fa8b9ca8f9017f) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:20.168729 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f561fe-95f9-4c36-b3b9-685ade2a2724 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:20.170374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f561fe-95f9-4c36-b3b9-685ade2a2724 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:20.176654 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 183/376] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Jul 20 15:12:20 2022] GET /identity/v3/role_assignments?group.id=921a4e3d2fdf49ba82cc42dc6f7f9737&scope.project.id=b271a003871a42fb99fa8b9ca8f9017f&role.id=2da8d544efe44ce6a41160d5759be402 => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:12:20.204259 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38524e60-0030-4652-ab67-3d98687d1569 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:20.205756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38524e60-0030-4652-ab67-3d98687d1569 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:20.284724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:20.322944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 194/377] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:20.329454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f561fe-95f9-4c36-b3b9-685ade2a2724 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:20.394931 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:20.479142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f561fe-95f9-4c36-b3b9-685ade2a2724 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:20.529469 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30b4a9-a5c0-4c06-af33-9cd1b33f4b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:20.530371 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30b4a9-a5c0-4c06-af33-9cd1b33f4b8e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:20.530653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30b4a9-a5c0-4c06-af33-9cd1b33f4b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:20.530930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30b4a9-a5c0-4c06-af33-9cd1b33f4b8e None admin] PATH_INFO: `/v3/projects/effa579779b9494f8b1dd5831edaae25` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:20.534209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30b4a9-a5c0-4c06-af33-9cd1b33f4b8e None admin] RBAC: Authorizing `identity:get_project(project_id=effa579779b9494f8b1dd5831edaae25)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:20.535782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30b4a9-a5c0-4c06-af33-9cd1b33f4b8e None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:20.538356 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 184/378] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:20 2022] GET /identity/v3/projects/effa579779b9494f8b1dd5831edaae25 => generated 344 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:20.543593 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38524e60-0030-4652-ab67-3d98687d1569 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:20.667127 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38524e60-0030-4652-ab67-3d98687d1569 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:20.710356 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db591e1a-a8fd-4b90-a351-cf0b8677dcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:20.711228 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591e1a-a8fd-4b90-a351-cf0b8677dcaa None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:20.711506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591e1a-a8fd-4b90-a351-cf0b8677dcaa None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:20.711777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591e1a-a8fd-4b90-a351-cf0b8677dcaa None admin] PATH_INFO: `/v3/groups/2e08bdaa12454430a3462f84c00cb89f` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:20.721411 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db591e1a-a8fd-4b90-a351-cf0b8677dcaa None admin] RBAC: Authorizing `identity:get_group(group_id=2e08bdaa12454430a3462f84c00cb89f)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:20.722960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db591e1a-a8fd-4b90-a351-cf0b8677dcaa None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:20.725576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 195/379] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Jul 20 15:12:20 2022] GET /identity/v3/groups/2e08bdaa12454430a3462f84c00cb89f => generated 229 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:20.731070 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30b4a9-a5c0-4c06-af33-9cd1b33f4b8e None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:20.873029 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30b4a9-a5c0-4c06-af33-9cd1b33f4b8e None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:20.887335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:20.898993 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1333434-8fbd-4c35-9473-f3b7f9d39dd9 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:20.899266 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1333434-8fbd-4c35-9473-f3b7f9d39dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:20.899541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1333434-8fbd-4c35-9473-f3b7f9d39dd9 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:20.901862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 196/380] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:12:20.906279 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b374d-ca63-4c67-8890-c135aed23d00 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:20.906600 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b374d-ca63-4c67-8890-c135aed23d00 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:20.906870 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b374d-ca63-4c67-8890-c135aed23d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:20.913914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94fb56-f653-43e9-8d9b-107af541748b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:20.914628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94fb56-f653-43e9-8d9b-107af541748b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:20.914987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94fb56-f653-43e9-8d9b-107af541748b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:20.915259 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94fb56-f653-43e9-8d9b-107af541748b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:20.916156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94fb56-f653-43e9-8d9b-107af541748b None admin] RBAC: Adding query filter params (group.id=2e08bdaa12454430a3462f84c00cb89f, role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=effa579779b9494f8b1dd5831edaae25) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:20.916470 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94fb56-f653-43e9-8d9b-107af541748b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:20.917923 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94fb56-f653-43e9-8d9b-107af541748b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:20.926561 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 185/381] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Jul 20 15:12:20 2022] GET /identity/v3/role_assignments?group.id=2e08bdaa12454430a3462f84c00cb89f&scope.project.id=effa579779b9494f8b1dd5831edaae25&role.id=25516be1485c49538544ea8a596e7d17 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:20.929786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-b73b374d-ca63-4c67-8890-c135aed23d00 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:20.931382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b73b374d-ca63-4c67-8890-c135aed23d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:20.962675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:21.012395 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 197/382] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:12:21.036670 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94fb56-f653-43e9-8d9b-107af541748b None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:21.184814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94fb56-f653-43e9-8d9b-107af541748b None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:21.219555 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b9c64-cdf6-49cd-a492-039405f293b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:21.220361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b9c64-cdf6-49cd-a492-039405f293b6 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:21.220639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b9c64-cdf6-49cd-a492-039405f293b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:21.220913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b9c64-cdf6-49cd-a492-039405f293b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:21.232405 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b9c64-cdf6-49cd-a492-039405f293b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:21.241267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-518b9c64-cdf6-49cd-a492-039405f293b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:21.241267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:21.241267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:21.241267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:21.241267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:21.241267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:21.241267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:21.241267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:21.241267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:21.241267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:21.241267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:21.241267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:21.241267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:21.241267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:21.241267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:21.241267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:21.242172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:21.242172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:21.242172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:21.242172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:21.242172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:21.242172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:21.242172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:21.242172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:21.242172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:21.242172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:21.242172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:21.242172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:21.242172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:21.242172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:21.242172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:21.242172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:21.243018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:21.243018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:21.243018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:21.243018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:21.243018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:21.243018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:21.243018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:21.243018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:21.243018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:21.243018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:21.243018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:21.243018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:21.243018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 186/383] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:12:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:21.249925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b374d-ca63-4c67-8890-c135aed23d00 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:21.387101 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b374d-ca63-4c67-8890-c135aed23d00 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:21.422561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3a6ce-0159-4383-b52c-defcaa9907f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:21.423225 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3a6ce-0159-4383-b52c-defcaa9907f8 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:21.423503 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3a6ce-0159-4383-b52c-defcaa9907f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:21.423774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3a6ce-0159-4383-b52c-defcaa9907f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:21.424510 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3a6ce-0159-4383-b52c-defcaa9907f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:21.424812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3a6ce-0159-4383-b52c-defcaa9907f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:21.426309 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3a6ce-0159-4383-b52c-defcaa9907f8 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:21.436152 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 198/384] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:21.443188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b9c64-cdf6-49cd-a492-039405f293b6 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:21.488622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:21.510784 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65205d2a-eb18-40c9-b8c1-4ce5b98fd074 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:21.511117 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65205d2a-eb18-40c9-b8c1-4ce5b98fd074 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:21.511329 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65205d2a-eb18-40c9-b8c1-4ce5b98fd074 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:21.513106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 199/385] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:21.521293 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b64169-90c1-4e35-b18a-3eb9fbdb23bf None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:21.521577 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b64169-90c1-4e35-b18a-3eb9fbdb23bf None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:21.521855 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b64169-90c1-4e35-b18a-3eb9fbdb23bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:21.543173 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-36b64169-90c1-4e35-b18a-3eb9fbdb23bf None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:21.545264 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-36b64169-90c1-4e35-b18a-3eb9fbdb23bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:21.587810 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:21.621612 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b9c64-cdf6-49cd-a492-039405f293b6 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:21.660469 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 200/386] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:12:21.682335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b64169-90c1-4e35-b18a-3eb9fbdb23bf None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:21.699592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c18a15-9c12-42d6-8247-cba7169d23b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:21.700876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c18a15-9c12-42d6-8247-cba7169d23b9 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:21.701390 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c18a15-9c12-42d6-8247-cba7169d23b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:21.703624 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c18a15-9c12-42d6-8247-cba7169d23b9 None admin] PATH_INFO: `/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:21.706071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c18a15-9c12-42d6-8247-cba7169d23b9 None admin] RBAC: Authorizing `identity:get_group(group_id=921a4e3d2fdf49ba82cc42dc6f7f9737)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:21.709211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c18a15-9c12-42d6-8247-cba7169d23b9 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:21.712261 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 187/387] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Jul 20 15:12:21 2022] GET /identity/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:21.722175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c18a15-9c12-42d6-8247-cba7169d23b9 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:21.827083 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b64169-90c1-4e35-b18a-3eb9fbdb23bf None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:21.862770 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedac34-6f08-4ccd-a5f9-1fc0d9aa13b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:21.863573 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedac34-6f08-4ccd-a5f9-1fc0d9aa13b6 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:21.863855 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedac34-6f08-4ccd-a5f9-1fc0d9aa13b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:21.864155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedac34-6f08-4ccd-a5f9-1fc0d9aa13b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:21.875047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fedac34-6f08-4ccd-a5f9-1fc0d9aa13b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:21.883900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-8fedac34-6f08-4ccd-a5f9-1fc0d9aa13b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:21.883900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:21.883900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:21.883900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:21.883900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:21.883900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:21.883900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:21.883900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:21.883900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:21.883900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:21.883900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:21.883900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:21.883900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:21.883900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:21.883900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:21.883900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:21.884816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:21.884816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:21.884816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:21.884816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:21.884816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:21.884816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:21.884816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:21.884816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:21.884816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:21.884816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:21.884816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:21.884816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:21.884816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:21.884816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:21.884816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:21.884816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:21.885669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:21.885669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:21.885669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:21.885669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:21.885669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:21.885669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:21.885669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:21.885669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:21.885669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:21.885669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:21.885669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:21.885669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:21.885669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 201/388] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:21.896699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedac34-6f08-4ccd-a5f9-1fc0d9aa13b6 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:21.906138 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c18a15-9c12-42d6-8247-cba7169d23b9 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:21.914295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:21.936437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:21.975184 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f088028-d929-415d-906c-24cd2990f2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:21.976357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f088028-d929-415d-906c-24cd2990f2af None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:21.976838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f088028-d929-415d-906c-24cd2990f2af None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:21.977330 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f088028-d929-415d-906c-24cd2990f2af None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:21.978854 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f088028-d929-415d-906c-24cd2990f2af None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:21.980483 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f088028-d929-415d-906c-24cd2990f2af None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:21.986699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 188/389] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:21 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:21.993888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f088028-d929-415d-906c-24cd2990f2af None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:22.043182 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedac34-6f08-4ccd-a5f9-1fc0d9aa13b6 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:22.084747 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe5ccc-7d14-4f31-93c6-a2c3062fb0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:22.085439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe5ccc-7d14-4f31-93c6-a2c3062fb0db None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:22.085727 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe5ccc-7d14-4f31-93c6-a2c3062fb0db None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:22.086010 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe5ccc-7d14-4f31-93c6-a2c3062fb0db None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:22.086791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe5ccc-7d14-4f31-93c6-a2c3062fb0db None admin] RBAC: Adding query filter params (name=member) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:22.087103 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe5ccc-7d14-4f31-93c6-a2c3062fb0db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:22.088502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe5ccc-7d14-4f31-93c6-a2c3062fb0db None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:22.106145 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 202/390] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:12:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:22.111169 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe5ccc-7d14-4f31-93c6-a2c3062fb0db None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:22.197313 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f088028-d929-415d-906c-24cd2990f2af None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:22.244751 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:22.263715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519cdf6-d050-47b9-b920-2f09f2471ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:22.264968 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519cdf6-d050-47b9-b920-2f09f2471ed6 None admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:22.265533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519cdf6-d050-47b9-b920-2f09f2471ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:22.266792 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519cdf6-d050-47b9-b920-2f09f2471ed6 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82/groups/921a4e3d2fdf49ba82cc42dc6f7f9737/roles/2da8d544efe44ce6a41160d5759be402` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:22.271218 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519cdf6-d050-47b9-b920-2f09f2471ed6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba23663aa02d442ab621813797163d82, group_id=921a4e3d2fdf49ba82cc42dc6f7f9737, role_id=2da8d544efe44ce6a41160d5759be402)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:22.273043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519cdf6-d050-47b9-b920-2f09f2471ed6 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:22.284157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe5ccc-7d14-4f31-93c6-a2c3062fb0db None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:22.325507 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7af83-4657-438b-9266-71e835ab1787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:22.326883 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7af83-4657-438b-9266-71e835ab1787 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:22.327404 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7af83-4657-438b-9266-71e835ab1787 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:22.327847 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7af83-4657-438b-9266-71e835ab1787 None admin] PATH_INFO: `/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:22.329288 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7af83-4657-438b-9266-71e835ab1787 None admin] RBAC: Authorizing `identity:get_group(group_id=921a4e3d2fdf49ba82cc42dc6f7f9737)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:22.330995 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7af83-4657-438b-9266-71e835ab1787 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:22.333534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 203/391] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Jul 20 15:12:22 2022] GET /identity/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:22.338735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7af83-4657-438b-9266-71e835ab1787 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:22.357937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 189/392] 10.222.0.78 () {58 vars in 1442 bytes} [Wed Jul 20 15:12:21 2022] PUT /identity/v3/projects/ba23663aa02d442ab621813797163d82/groups/921a4e3d2fdf49ba82cc42dc6f7f9737/roles/2da8d544efe44ce6a41160d5759be402 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:12:22.457016 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7af83-4657-438b-9266-71e835ab1787 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:22.493174 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3e572-ba6c-4feb-b246-5df7321e8c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:22.494523 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3e572-ba6c-4feb-b246-5df7321e8c6c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:22.495056 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3e572-ba6c-4feb-b246-5df7321e8c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:22.495541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3e572-ba6c-4feb-b246-5df7321e8c6c None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:22.496925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3e572-ba6c-4feb-b246-5df7321e8c6c None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:22.498668 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3e572-ba6c-4feb-b246-5df7321e8c6c None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:22.500853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 204/393] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:22 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:12:22.507313 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519cdf6-d050-47b9-b920-2f09f2471ed6 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:22.565144 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:22.666456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519cdf6-d050-47b9-b920-2f09f2471ed6 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:22.704464 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1aabe6-f832-4ef1-bdc0-e95519816e36 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:22.704650 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510874df-273d-4a01-92ce-4c0aa401eccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:22.705210 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1aabe6-f832-4ef1-bdc0-e95519816e36 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:22.705493 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510874df-273d-4a01-92ce-4c0aa401eccc None admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:22.705773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510874df-273d-4a01-92ce-4c0aa401eccc None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:22.706045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510874df-273d-4a01-92ce-4c0aa401eccc None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82/groups/921a4e3d2fdf49ba82cc42dc6f7f9737/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:22.706333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1aabe6-f832-4ef1-bdc0-e95519816e36 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:22.708667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510874df-273d-4a01-92ce-4c0aa401eccc None admin] RBAC: Authorizing `identity:create_grant(project_id=ba23663aa02d442ab621813797163d82, group_id=921a4e3d2fdf49ba82cc42dc6f7f9737, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:22.709265 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 205/394] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:12:22.712704 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510874df-273d-4a01-92ce-4c0aa401eccc None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:22.714447 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa794e-d557-45f0-88ec-1ad8d42aa0af None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:22.714874 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa794e-d557-45f0-88ec-1ad8d42aa0af None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:22.715439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa794e-d557-45f0-88ec-1ad8d42aa0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:22.734623 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-dbfa794e-d557-45f0-88ec-1ad8d42aa0af None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:22.735896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-dbfa794e-d557-45f0-88ec-1ad8d42aa0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:22.746960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:22.793737 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 190/395] 10.222.0.78 () {58 vars in 1442 bytes} [Wed Jul 20 15:12:22 2022] PUT /identity/v3/projects/ba23663aa02d442ab621813797163d82/groups/921a4e3d2fdf49ba82cc42dc6f7f9737/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:12:22.797833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aff6ec-2875-43c8-b883-795c5ec91883 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:22.797833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aff6ec-2875-43c8-b883-795c5ec91883 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:22.797833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aff6ec-2875-43c8-b883-795c5ec91883 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:22.797833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 191/396] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:22.806287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a5658-9b48-4c24-bf49-86cbbe515d7d None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:22.806287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a5658-9b48-4c24-bf49-86cbbe515d7d None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:22.806287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a5658-9b48-4c24-bf49-86cbbe515d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:22.834834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-b67a5658-9b48-4c24-bf49-86cbbe515d7d None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:22.836159 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b67a5658-9b48-4c24-bf49-86cbbe515d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:22.867711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 206/397] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:22.903242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa794e-d557-45f0-88ec-1ad8d42aa0af None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:22.919905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 192/398] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:12:22.947110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a5658-9b48-4c24-bf49-86cbbe515d7d None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:22.982738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:23.066813 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a5658-9b48-4c24-bf49-86cbbe515d7d None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:23.083624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa794e-d557-45f0-88ec-1ad8d42aa0af None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:23.111480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a008b-bcd1-498e-bf56-674b4b906ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:23.113268 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a008b-bcd1-498e-bf56-674b4b906ddb None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:23.113625 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a008b-bcd1-498e-bf56-674b4b906ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:23.114648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a008b-bcd1-498e-bf56-674b4b906ddb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:23.131643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a008b-bcd1-498e-bf56-674b4b906ddb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:23.135612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d255016-c06e-4934-b66f-c0c712cbc7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:23.136631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d255016-c06e-4934-b66f-c0c712cbc7cc None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:23.140115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d255016-c06e-4934-b66f-c0c712cbc7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:23.140115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d255016-c06e-4934-b66f-c0c712cbc7cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:23.141431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-a42a008b-bcd1-498e-bf56-674b4b906ddb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:23.141431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:23.141431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:23.141431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:23.141431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:23.141431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:23.141431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:23.141431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:23.141431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:23.141431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:23.141431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:23.141431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:23.141431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:23.141431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:23.141431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:23.141431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:23.142374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:23.142374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:23.142374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:23.142374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:23.142374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:23.142374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:23.142374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:23.142374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:23.142374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:23.142374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:23.142374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:23.142374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:23.142374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:23.142374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:23.142374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:23.142374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:23.143195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:23.143195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:23.143195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:23.143195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:23.143195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:23.143195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:23.143195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:23.143195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:23.143195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:23.143195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:23.143195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:23.143195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:23.143195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 193/399] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:12:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:23.148683 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a008b-bcd1-498e-bf56-674b4b906ddb None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:23.153624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d255016-c06e-4934-b66f-c0c712cbc7cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:23.167653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-6d255016-c06e-4934-b66f-c0c712cbc7cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:23.167653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:23.167653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:23.167653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:23.167653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:23.167653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:23.167653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:23.167653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:23.167653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:23.167653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:23.167653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:23.167653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:23.167653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:23.167653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:23.167653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:23.167653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:23.169033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:23.169033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:23.169033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:23.169033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:23.169033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:23.169033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:23.169033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:23.169033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:23.169033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:23.169033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:23.169033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:23.169033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:23.169033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:23.169033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:23.169033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:23.169033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:23.170066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:23.170066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:23.170066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:23.170066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:23.170066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:23.170066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:23.170066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:23.170066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:23.170066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:23.170066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:23.170066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:23.170066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:23.173069 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 207/400] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:23.179432 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d255016-c06e-4934-b66f-c0c712cbc7cc None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:23.185242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:23.307215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:23.327004 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a008b-bcd1-498e-bf56-674b4b906ddb None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:23.354556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d255016-c06e-4934-b66f-c0c712cbc7cc None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:23.365767 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a382c-035f-42af-b846-84fa03a0a961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:23.366795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a382c-035f-42af-b846-84fa03a0a961 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:23.367264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a382c-035f-42af-b846-84fa03a0a961 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:23.367717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a382c-035f-42af-b846-84fa03a0a961 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:23.368651 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a382c-035f-42af-b846-84fa03a0a961 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:23.369198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a382c-035f-42af-b846-84fa03a0a961 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:23.370967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a382c-035f-42af-b846-84fa03a0a961 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:23.381208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 194/401] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:12:23.387134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a382c-035f-42af-b846-84fa03a0a961 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:23.413820 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6454dc2-2365-4175-9265-2dd22bc61f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:23.414524 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6454dc2-2365-4175-9265-2dd22bc61f13 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:23.414807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6454dc2-2365-4175-9265-2dd22bc61f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:23.415080 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6454dc2-2365-4175-9265-2dd22bc61f13 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:23.415824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6454dc2-2365-4175-9265-2dd22bc61f13 None admin] RBAC: Adding query filter params (name=member) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:23.416129 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6454dc2-2365-4175-9265-2dd22bc61f13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:23.417504 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6454dc2-2365-4175-9265-2dd22bc61f13 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:23.427146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 208/402] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:12:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:12:23.435967 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6454dc2-2365-4175-9265-2dd22bc61f13 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:23.502644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a382c-035f-42af-b846-84fa03a0a961 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:23.539008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd6806-bbcf-4591-840a-730ca716d987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:23.540622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd6806-bbcf-4591-840a-730ca716d987 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:23.541113 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd6806-bbcf-4591-840a-730ca716d987 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:23.541580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd6806-bbcf-4591-840a-730ca716d987 None admin] PATH_INFO: `/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:23.543028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd6806-bbcf-4591-840a-730ca716d987 None admin] RBAC: Authorizing `identity:get_group(group_id=921a4e3d2fdf49ba82cc42dc6f7f9737)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:23.544674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd6806-bbcf-4591-840a-730ca716d987 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:23.546919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 195/403] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Jul 20 15:12:23 2022] GET /identity/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:23.553059 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd6806-bbcf-4591-840a-730ca716d987 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:23.569635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:23.607504 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:23.659943 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd6806-bbcf-4591-840a-730ca716d987 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:23.692415 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6454dc2-2365-4175-9265-2dd22bc61f13 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:23.702523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef965c01-8e8d-41ba-b711-60c511fc985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:23.703390 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef965c01-8e8d-41ba-b711-60c511fc985a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:23.703676 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef965c01-8e8d-41ba-b711-60c511fc985a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:23.703952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef965c01-8e8d-41ba-b711-60c511fc985a None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:23.705165 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef965c01-8e8d-41ba-b711-60c511fc985a None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:23.706673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef965c01-8e8d-41ba-b711-60c511fc985a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:23.709424 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 196/404] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:23 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:12:23.716426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef965c01-8e8d-41ba-b711-60c511fc985a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:23.756063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c382e7-7d0e-43a8-95e8-9e22c0ae357c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:23.757261 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c382e7-7d0e-43a8-95e8-9e22c0ae357c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:23.757549 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c382e7-7d0e-43a8-95e8-9e22c0ae357c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:23.757830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c382e7-7d0e-43a8-95e8-9e22c0ae357c None admin] PATH_INFO: `/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:23.759329 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c382e7-7d0e-43a8-95e8-9e22c0ae357c None admin] RBAC: Authorizing `identity:get_group(group_id=921a4e3d2fdf49ba82cc42dc6f7f9737)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:23.761246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c382e7-7d0e-43a8-95e8-9e22c0ae357c None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:23.763507 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 209/405] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Jul 20 15:12:23 2022] GET /identity/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737 => generated 226 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:23.767132 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdd3f7-a954-471e-8143-7b1e7218cc4f None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:23.767650 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdd3f7-a954-471e-8143-7b1e7218cc4f None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:23.768167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdd3f7-a954-471e-8143-7b1e7218cc4f None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:23.769664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 210/406] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:23.771967 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cdd3f7-a954-471e-8143-7b1e7218cc4f None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:23.829351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:23.859885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef965c01-8e8d-41ba-b711-60c511fc985a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:23.896874 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d313d1f-581a-40ad-82fe-773df576d714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:23.897764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d313d1f-581a-40ad-82fe-773df576d714 None admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:23.898071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d313d1f-581a-40ad-82fe-773df576d714 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:23.898461 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d313d1f-581a-40ad-82fe-773df576d714 None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f/groups/921a4e3d2fdf49ba82cc42dc6f7f9737/roles/2da8d544efe44ce6a41160d5759be402` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:23.899986 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cdd3f7-a954-471e-8143-7b1e7218cc4f None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:23.900831 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d313d1f-581a-40ad-82fe-773df576d714 None admin] RBAC: Authorizing `identity:create_grant(project_id=b271a003871a42fb99fa8b9ca8f9017f, group_id=921a4e3d2fdf49ba82cc42dc6f7f9737, role_id=2da8d544efe44ce6a41160d5759be402)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:23.902342 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d313d1f-581a-40ad-82fe-773df576d714 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:23.937463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e240a0-61f8-4c6f-8ef2-0993cf7c06ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:23.938688 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e240a0-61f8-4c6f-8ef2-0993cf7c06ff None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:23.939196 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e240a0-61f8-4c6f-8ef2-0993cf7c06ff None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:23.939723 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e240a0-61f8-4c6f-8ef2-0993cf7c06ff None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:23.941081 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e240a0-61f8-4c6f-8ef2-0993cf7c06ff None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:23.943446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e240a0-61f8-4c6f-8ef2-0993cf7c06ff None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:23.945405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 211/407] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:23 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:23.946756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ce70c-cbe8-4235-a654-d5ba5793a5f0 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:23.947262 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ce70c-cbe8-4235-a654-d5ba5793a5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:23.947740 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ce70c-cbe8-4235-a654-d5ba5793a5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:23.953656 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:23.966732 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-2d9ce70c-cbe8-4235-a654-d5ba5793a5f0 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:23.967909 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2d9ce70c-cbe8-4235-a654-d5ba5793a5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:24.016102 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 197/408] 10.222.0.78 () {58 vars in 1442 bytes} [Wed Jul 20 15:12:23 2022] PUT /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f/groups/921a4e3d2fdf49ba82cc42dc6f7f9737/roles/2da8d544efe44ce6a41160d5759be402 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:12:24.017126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d313d1f-581a-40ad-82fe-773df576d714 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:24.056563 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 212/409] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:24.086374 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ce70c-cbe8-4235-a654-d5ba5793a5f0 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:24.214318 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d313d1f-581a-40ad-82fe-773df576d714 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:24.250565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ce70c-cbe8-4235-a654-d5ba5793a5f0 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:24.258992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4211876-db1f-4699-bf9c-47c5fff33729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:24.260002 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4211876-db1f-4699-bf9c-47c5fff33729 None admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:24.260291 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4211876-db1f-4699-bf9c-47c5fff33729 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:24.260565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4211876-db1f-4699-bf9c-47c5fff33729 None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f/groups/921a4e3d2fdf49ba82cc42dc6f7f9737/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:24.262774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4211876-db1f-4699-bf9c-47c5fff33729 None admin] RBAC: Authorizing `identity:create_grant(project_id=b271a003871a42fb99fa8b9ca8f9017f, group_id=921a4e3d2fdf49ba82cc42dc6f7f9737, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:24.264630 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4211876-db1f-4699-bf9c-47c5fff33729 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:24.287439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:24.292841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67751d-7b43-47ec-a3f6-f1b22f64946c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:24.293779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67751d-7b43-47ec-a3f6-f1b22f64946c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:24.294064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67751d-7b43-47ec-a3f6-f1b22f64946c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:24.294549 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67751d-7b43-47ec-a3f6-f1b22f64946c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:24.305846 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67751d-7b43-47ec-a3f6-f1b22f64946c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:24.313862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-3a67751d-7b43-47ec-a3f6-f1b22f64946c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:12:24.313862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:24.313862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:24.313862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:24.313862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:24.313862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:24.313862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:24.313862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:24.313862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:24.313862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:24.313862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:24.313862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:24.313862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:24.313862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:24.313862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:24.313862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:24.368797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:24.368797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:24.368797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:24.368797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:24.368797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:24.368797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:24.368797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:24.368797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:24.368797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:24.368797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:24.368797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:24.368797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:24.368797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:24.368797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:24.368797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:24.368797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:24.369614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:24.369614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:24.369614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:24.369614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:24.369614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:24.369614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:24.369614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:24.369614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:24.369614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:24.369614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:24.369614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:12:24.369614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:24.369614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 213/410] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:12:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:24.369614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67751d-7b43-47ec-a3f6-f1b22f64946c None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:24.399344 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:24.409857 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 198/411] 10.222.0.78 () {58 vars in 1442 bytes} [Wed Jul 20 15:12:24 2022] PUT /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f/groups/921a4e3d2fdf49ba82cc42dc6f7f9737/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:12:24.439745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67751d-7b43-47ec-a3f6-f1b22f64946c None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:24.484789 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24ee4f-d0ff-4795-ac74-c77c38c0b0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:24.485406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24ee4f-d0ff-4795-ac74-c77c38c0b0cf None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:24.485787 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24ee4f-d0ff-4795-ac74-c77c38c0b0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:24.486166 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24ee4f-d0ff-4795-ac74-c77c38c0b0cf None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:24.487039 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24ee4f-d0ff-4795-ac74-c77c38c0b0cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:24.487365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24ee4f-d0ff-4795-ac74-c77c38c0b0cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:24.489678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24ee4f-d0ff-4795-ac74-c77c38c0b0cf None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:24.502351 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 214/412] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:24.509810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4211876-db1f-4699-bf9c-47c5fff33729 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:24.628785 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4211876-db1f-4699-bf9c-47c5fff33729 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:24.665758 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622060b8-6aa2-4820-b7ba-50d47d233580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:24.667006 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622060b8-6aa2-4820-b7ba-50d47d233580 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:24.667495 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622060b8-6aa2-4820-b7ba-50d47d233580 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:24.667996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622060b8-6aa2-4820-b7ba-50d47d233580 None admin] PATH_INFO: `/v3/groups/2e08bdaa12454430a3462f84c00cb89f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:24.669398 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622060b8-6aa2-4820-b7ba-50d47d233580 None admin] RBAC: Authorizing `identity:get_group(group_id=2e08bdaa12454430a3462f84c00cb89f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:24.671085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622060b8-6aa2-4820-b7ba-50d47d233580 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:24.673227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 199/413] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Jul 20 15:12:24 2022] GET /identity/v3/groups/2e08bdaa12454430a3462f84c00cb89f => generated 229 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:24.679230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24ee4f-d0ff-4795-ac74-c77c38c0b0cf None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:24.752965 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:24.817706 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24ee4f-d0ff-4795-ac74-c77c38c0b0cf None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:24.866427 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3add362-2466-4bfc-a274-2712476b29c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:24.867364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3add362-2466-4bfc-a274-2712476b29c1 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:24.867653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3add362-2466-4bfc-a274-2712476b29c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:24.867930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3add362-2466-4bfc-a274-2712476b29c1 None admin] PATH_INFO: `/v3/projects/effa579779b9494f8b1dd5831edaae25` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:24.869184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3add362-2466-4bfc-a274-2712476b29c1 None admin] RBAC: Authorizing `identity:get_project(project_id=effa579779b9494f8b1dd5831edaae25)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:24.870667 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3add362-2466-4bfc-a274-2712476b29c1 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:24.872539 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 215/414] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:24 2022] GET /identity/v3/projects/effa579779b9494f8b1dd5831edaae25 => generated 344 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:24.878121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622060b8-6aa2-4820-b7ba-50d47d233580 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:24.905815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf470a-94fd-4639-9e26-936e8059b139 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:24.906143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf470a-94fd-4639-9e26-936e8059b139 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:24.906453 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf470a-94fd-4639-9e26-936e8059b139 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:24.908116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 216/415] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:12:24.913943 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e5a0f-bb08-4158-b79a-3706f2348f38 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:24.914237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e5a0f-bb08-4158-b79a-3706f2348f38 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:24.914550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e5a0f-bb08-4158-b79a-3706f2348f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:24.932500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-739e5a0f-bb08-4158-b79a-3706f2348f38 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:24.933706 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-739e5a0f-bb08-4158-b79a-3706f2348f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:24.991841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:25.026985 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622060b8-6aa2-4820-b7ba-50d47d233580 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:25.044856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 217/416] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:12:25.062223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361065b-4a4c-45b9-82ad-38ac231e00cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:25.063138 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361065b-4a4c-45b9-82ad-38ac231e00cf None admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:25.063422 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361065b-4a4c-45b9-82ad-38ac231e00cf None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:25.063691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361065b-4a4c-45b9-82ad-38ac231e00cf None admin] PATH_INFO: `/v3/projects/effa579779b9494f8b1dd5831edaae25/groups/2e08bdaa12454430a3462f84c00cb89f/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:25.065637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d361065b-4a4c-45b9-82ad-38ac231e00cf None admin] RBAC: Authorizing `identity:create_grant(project_id=effa579779b9494f8b1dd5831edaae25, group_id=2e08bdaa12454430a3462f84c00cb89f, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:25.067112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d361065b-4a4c-45b9-82ad-38ac231e00cf None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:25.075102 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e5a0f-bb08-4158-b79a-3706f2348f38 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:25.141741 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:25.193458 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 200/417] 10.222.0.78 () {58 vars in 1442 bytes} [Wed Jul 20 15:12:24 2022] PUT /identity/v3/projects/effa579779b9494f8b1dd5831edaae25/groups/2e08bdaa12454430a3462f84c00cb89f/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:12:25.193994 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e5a0f-bb08-4158-b79a-3706f2348f38 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:25.234044 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b890738-3fa1-475e-bccd-15dfa3b6c1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:25.234909 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b890738-3fa1-475e-bccd-15dfa3b6c1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:25.235193 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b890738-3fa1-475e-bccd-15dfa3b6c1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:25.235471 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b890738-3fa1-475e-bccd-15dfa3b6c1b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:25.247521 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b890738-3fa1-475e-bccd-15dfa3b6c1b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:25.256364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-9b890738-3fa1-475e-bccd-15dfa3b6c1b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:25.256364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:25.256364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:25.256364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:25.256364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:25.256364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:25.256364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:25.256364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:25.256364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:25.256364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:25.256364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:25.256364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:25.256364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:25.256364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:25.256364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:25.256364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:25.257333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:25.257333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:25.257333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:25.257333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:25.257333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:25.257333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:25.257333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:25.257333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:25.257333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:25.257333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:25.257333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:25.257333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:25.257333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:25.257333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:25.257333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:25.257333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:25.258612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:25.258612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:25.258612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:25.258612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:25.258612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:25.258612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:25.258612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:25.258612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:25.258612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:25.258612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:25.258612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:25.258612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:25.258612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 218/418] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:12:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:25.263679 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361065b-4a4c-45b9-82ad-38ac231e00cf None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:25.414823 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15c3e6-ddd6-4b5d-9e48-2c45dc752c5f None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:25.415132 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15c3e6-ddd6-4b5d-9e48-2c45dc752c5f None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:25.415310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15c3e6-ddd6-4b5d-9e48-2c45dc752c5f None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:25.416430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361065b-4a4c-45b9-82ad-38ac231e00cf None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:25.417121 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 219/419] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:25.422702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982f8f7-f899-499a-8a1f-9f3d8ac70d1c None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:25.423318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982f8f7-f899-499a-8a1f-9f3d8ac70d1c None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:25.423791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982f8f7-f899-499a-8a1f-9f3d8ac70d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:25.448216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:25.460008 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-d982f8f7-f899-499a-8a1f-9f3d8ac70d1c None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:25.461283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d982f8f7-f899-499a-8a1f-9f3d8ac70d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:25.464984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4b78b-7934-4c98-8199-4c1a700aee44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:25.465675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4b78b-7934-4c98-8199-4c1a700aee44 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:25.465963 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4b78b-7934-4c98-8199-4c1a700aee44 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:25.466394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4b78b-7934-4c98-8199-4c1a700aee44 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:25.467058 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4b78b-7934-4c98-8199-4c1a700aee44 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:25.467436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4b78b-7934-4c98-8199-4c1a700aee44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:25.468910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4b78b-7934-4c98-8199-4c1a700aee44 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:25.488392 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 201/420] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:25.495179 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4b78b-7934-4c98-8199-4c1a700aee44 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:25.588918 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 220/421] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:25.612204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982f8f7-f899-499a-8a1f-9f3d8ac70d1c None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:25.622647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4b78b-7934-4c98-8199-4c1a700aee44 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:25.692178 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253bedd-8cbe-48a7-91f8-ab743f287615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:25.693543 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253bedd-8cbe-48a7-91f8-ab743f287615 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:25.693975 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253bedd-8cbe-48a7-91f8-ab743f287615 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:25.694426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253bedd-8cbe-48a7-91f8-ab743f287615 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:25.707130 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:25.774111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982f8f7-f899-499a-8a1f-9f3d8ac70d1c None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:25.811076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20fe26-d12d-4856-80a9-617a3da1e25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:25.812579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20fe26-d12d-4856-80a9-617a3da1e25c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:25.813015 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20fe26-d12d-4856-80a9-617a3da1e25c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:25.813467 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20fe26-d12d-4856-80a9-617a3da1e25c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:25.824743 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20fe26-d12d-4856-80a9-617a3da1e25c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:25.834342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-ef20fe26-d12d-4856-80a9-617a3da1e25c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:25.834342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:25.834342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:25.834342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:25.834342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:25.834342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:25.834342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:25.834342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:25.834342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:25.834342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:25.834342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:25.834342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:25.834342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:25.834342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:25.834342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:25.834342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:25.835473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:25.835473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:25.835473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:25.835473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:25.835473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:25.835473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:25.835473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:25.835473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:25.835473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:25.835473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:25.835473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:25.835473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:25.835473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:25.835473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:25.835473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:25.835473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:25.836480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:25.836480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:25.836480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:25.836480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:25.836480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:25.836480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:25.836480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:25.836480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:25.836480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:25.836480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:25.836480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:25.836480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:25.838612 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:25.839243 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 221/422] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:25.851244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20fe26-d12d-4856-80a9-617a3da1e25c None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:25.853266 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253bedd-8cbe-48a7-91f8-ab743f287615 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:25.854966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253bedd-8cbe-48a7-91f8-ab743f287615 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:26.002699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20fe26-d12d-4856-80a9-617a3da1e25c None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:26.004468 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:26.056668 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 202/423] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 15:12:26.062839 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253bedd-8cbe-48a7-91f8-ab743f287615 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:26.078521 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da09d6f-8877-4f88-b65a-d11992011bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:26.079203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da09d6f-8877-4f88-b65a-d11992011bed None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:26.079492 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da09d6f-8877-4f88-b65a-d11992011bed None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:26.079793 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da09d6f-8877-4f88-b65a-d11992011bed None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:26.080551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da09d6f-8877-4f88-b65a-d11992011bed None admin] RBAC: Adding query filter params (name=member) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:26.080864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da09d6f-8877-4f88-b65a-d11992011bed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:26.082292 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da09d6f-8877-4f88-b65a-d11992011bed None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:26.103142 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 222/424] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:12:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:26.110401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da09d6f-8877-4f88-b65a-d11992011bed None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:26.195230 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:26.206682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253bedd-8cbe-48a7-91f8-ab743f287615 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:26.246407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fe075-e9b1-432a-9cc5-8c12a033efd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:26.247655 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fe075-e9b1-432a-9cc5-8c12a033efd9 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:26.247951 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fe075-e9b1-432a-9cc5-8c12a033efd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:26.248237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fe075-e9b1-432a-9cc5-8c12a033efd9 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:26.249457 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fe075-e9b1-432a-9cc5-8c12a033efd9 None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:26.250955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fe075-e9b1-432a-9cc5-8c12a033efd9 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:26.253734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 203/425] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:26 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:26.259451 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fe075-e9b1-432a-9cc5-8c12a033efd9 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:26.375870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fe075-e9b1-432a-9cc5-8c12a033efd9 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:26.387959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da09d6f-8877-4f88-b65a-d11992011bed None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:26.424180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434c55f-b765-428f-bf71-343441d7427c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:26.425046 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434c55f-b765-428f-bf71-343441d7427c None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:26.425329 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434c55f-b765-428f-bf71-343441d7427c None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:26.425607 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434c55f-b765-428f-bf71-343441d7427c None admin] PATH_INFO: `/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:26.427857 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434c55f-b765-428f-bf71-343441d7427c None admin] RBAC: Authorizing `identity:get_group(group_id=921a4e3d2fdf49ba82cc42dc6f7f9737)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:26.429306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434c55f-b765-428f-bf71-343441d7427c None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:26.433893 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 204/426] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Jul 20 15:12:26 2022] GET /identity/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:26.435068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c9d07-5dee-4571-b9c0-8b577a678034 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:26.435355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c9d07-5dee-4571-b9c0-8b577a678034 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:26.435632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c9d07-5dee-4571-b9c0-8b577a678034 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:26.437787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 205/427] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:26.438690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c9d07-5dee-4571-b9c0-8b577a678034 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:26.452554 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4997c7a-1e1d-4ee9-bc03-bc0c25c01d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:26.453803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4997c7a-1e1d-4ee9-bc03-bc0c25c01d08 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:26.454331 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4997c7a-1e1d-4ee9-bc03-bc0c25c01d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:26.454807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4997c7a-1e1d-4ee9-bc03-bc0c25c01d08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:26.473358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:26.490635 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:26.526894 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4997c7a-1e1d-4ee9-bc03-bc0c25c01d08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:26.528965 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4997c7a-1e1d-4ee9-bc03-bc0c25c01d08 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:26.573346 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c9d07-5dee-4571-b9c0-8b577a678034 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:26.618314 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432ff61-9c5c-44c7-bf53-48620e055a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:26.618990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432ff61-9c5c-44c7-bf53-48620e055a75 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:26.619278 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432ff61-9c5c-44c7-bf53-48620e055a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:26.619557 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432ff61-9c5c-44c7-bf53-48620e055a75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:26.620836 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432ff61-9c5c-44c7-bf53-48620e055a75 None admin] RBAC: Adding query filter params (group.id=921a4e3d2fdf49ba82cc42dc6f7f9737, role.id=2da8d544efe44ce6a41160d5759be402, scope.project.id=ba23663aa02d442ab621813797163d82) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:26.621157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432ff61-9c5c-44c7-bf53-48620e055a75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:26.622927 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432ff61-9c5c-44c7-bf53-48620e055a75 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:26.632492 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 206/428] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Jul 20 15:12:26 2022] GET /identity/v3/role_assignments?group.id=921a4e3d2fdf49ba82cc42dc6f7f9737&scope.project.id=ba23663aa02d442ab621813797163d82&role.id=2da8d544efe44ce6a41160d5759be402 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:26.633959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e760fb-db97-42b7-ba81-253e68b4f982 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:26.634443 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e760fb-db97-42b7-ba81-253e68b4f982 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:26.634871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e760fb-db97-42b7-ba81-253e68b4f982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:26.635191 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 223/429] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:26.650507 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4997c7a-1e1d-4ee9-bc03-bc0c25c01d08 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:26.675442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-18e760fb-db97-42b7-ba81-253e68b4f982 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:26.677397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-18e760fb-db97-42b7-ba81-253e68b4f982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:26.726704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:26.773010 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4997c7a-1e1d-4ee9-bc03-bc0c25c01d08 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:26.810785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308d54e-b153-4956-8501-66442a1207b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:26.812027 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308d54e-b153-4956-8501-66442a1207b8 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:26.814792 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308d54e-b153-4956-8501-66442a1207b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:26.815307 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308d54e-b153-4956-8501-66442a1207b8 None admin] PATH_INFO: `/v3/projects/ba23663aa02d442ab621813797163d82` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:26.816898 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308d54e-b153-4956-8501-66442a1207b8 None admin] RBAC: Authorizing `identity:get_project(project_id=ba23663aa02d442ab621813797163d82)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:26.819788 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308d54e-b153-4956-8501-66442a1207b8 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:26.822342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 224/430] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:26 2022] GET /identity/v3/projects/ba23663aa02d442ab621813797163d82 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:26.827689 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308d54e-b153-4956-8501-66442a1207b8 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:26.866494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 207/431] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:26.892154 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e760fb-db97-42b7-ba81-253e68b4f982 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:26.916110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:26.953680 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308d54e-b153-4956-8501-66442a1207b8 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:26.985515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:26.991747 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79239a3b-49e9-4fc2-a83a-da293b093624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:26.992796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79239a3b-49e9-4fc2-a83a-da293b093624 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:26.993230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79239a3b-49e9-4fc2-a83a-da293b093624 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:26.993674 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79239a3b-49e9-4fc2-a83a-da293b093624 None admin] PATH_INFO: `/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:26.995824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79239a3b-49e9-4fc2-a83a-da293b093624 None admin] RBAC: Authorizing `identity:get_group(group_id=921a4e3d2fdf49ba82cc42dc6f7f9737)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:26.997548 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79239a3b-49e9-4fc2-a83a-da293b093624 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:26.999835 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 225/432] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Jul 20 15:12:26 2022] GET /identity/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:27.004948 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02987b91-ed00-4e0b-8c1e-f7185dc906f2 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:27.005518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02987b91-ed00-4e0b-8c1e-f7185dc906f2 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:27.005947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02987b91-ed00-4e0b-8c1e-f7185dc906f2 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:27.007677 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 226/433] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:27.008472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e760fb-db97-42b7-ba81-253e68b4f982 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:27.008601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02987b91-ed00-4e0b-8c1e-f7185dc906f2 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:27.049031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315d10d-16fe-498d-ad8d-9997a335c48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:27.049893 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315d10d-16fe-498d-ad8d-9997a335c48b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:27.050183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315d10d-16fe-498d-ad8d-9997a335c48b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:27.050509 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315d10d-16fe-498d-ad8d-9997a335c48b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:27.061714 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3315d10d-16fe-498d-ad8d-9997a335c48b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:27.069802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-3315d10d-16fe-498d-ad8d-9997a335c48b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:27.069802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:27.069802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:27.069802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:27.069802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:27.069802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:27.069802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:27.069802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:27.069802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:27.069802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:27.069802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:27.069802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:27.069802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:27.069802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:27.069802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:27.069802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:27.070749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:27.070749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:27.070749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:27.070749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:27.070749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:27.070749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:27.070749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:27.070749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:27.070749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:27.070749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:27.070749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:27.070749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:27.070749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:27.070749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:27.070749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:27.070749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:27.075294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:27.075294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:27.075294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:27.075294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:27.075294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:27.075294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:27.075294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:27.075294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:27.075294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:27.075294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:27.075294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:27.075294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:27.075294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 208/434] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:12:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:27.075294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a42e39-2bfe-4329-9ae7-40379c16e04f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:27.076222 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a42e39-2bfe-4329-9ae7-40379c16e04f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:27.076222 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a42e39-2bfe-4329-9ae7-40379c16e04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:27.095001 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-c6a42e39-2bfe-4329-9ae7-40379c16e04f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:27.096425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-c6a42e39-2bfe-4329-9ae7-40379c16e04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:27.128145 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02987b91-ed00-4e0b-8c1e-f7185dc906f2 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:27.165334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11e659-ebd6-4a15-96cf-202105bbf34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:27.165986 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11e659-ebd6-4a15-96cf-202105bbf34f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:27.170545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11e659-ebd6-4a15-96cf-202105bbf34f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:27.170692 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 209/435] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:27.170772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11e659-ebd6-4a15-96cf-202105bbf34f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:27.170772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11e659-ebd6-4a15-96cf-202105bbf34f None admin] RBAC: Adding query filter params (group.id=921a4e3d2fdf49ba82cc42dc6f7f9737, role.id=d85783606f6649f5b9465f407cafb610, scope.project.id=ba23663aa02d442ab621813797163d82) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:27.170929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a42e39-2bfe-4329-9ae7-40379c16e04f None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:27.171006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11e659-ebd6-4a15-96cf-202105bbf34f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:27.171006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11e659-ebd6-4a15-96cf-202105bbf34f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:27.180440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 227/436] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Jul 20 15:12:27 2022] GET /identity/v3/role_assignments?group.id=921a4e3d2fdf49ba82cc42dc6f7f9737&scope.project.id=ba23663aa02d442ab621813797163d82&role.id=d85783606f6649f5b9465f407cafb610 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:27.181751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:27.193774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11e659-ebd6-4a15-96cf-202105bbf34f None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:27.310719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:27.312221 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a42e39-2bfe-4329-9ae7-40379c16e04f None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:27.322233 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11e659-ebd6-4a15-96cf-202105bbf34f None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:27.362757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc55de9-ac81-429a-857b-7c575287d4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:27.363760 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc55de9-ac81-429a-857b-7c575287d4ab None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:27.364626 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc55de9-ac81-429a-857b-7c575287d4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:27.364925 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc55de9-ac81-429a-857b-7c575287d4ab None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:27.365654 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc55de9-ac81-429a-857b-7c575287d4ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:27.365959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc55de9-ac81-429a-857b-7c575287d4ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:27.367447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc55de9-ac81-429a-857b-7c575287d4ab None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:27.369258 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08e627-21c2-4616-abc7-1bc210f3e1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:27.370063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08e627-21c2-4616-abc7-1bc210f3e1aa None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:27.370376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08e627-21c2-4616-abc7-1bc210f3e1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:27.370657 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08e627-21c2-4616-abc7-1bc210f3e1aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:27.384296 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 210/437] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:27.385635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc55de9-ac81-429a-857b-7c575287d4ab None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:27.385797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08e627-21c2-4616-abc7-1bc210f3e1aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:27.393762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-0e08e627-21c2-4616-abc7-1bc210f3e1aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:27.393762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:27.393762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:27.393762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:27.393762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:27.393762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:27.393762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:27.393762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:27.393762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:27.393762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:27.393762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:27.393762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:27.393762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:27.393762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:27.393762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:27.393762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:27.394791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:27.394791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:27.394791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:27.394791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:27.394791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:27.394791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:27.394791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:27.394791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:27.394791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:27.394791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:27.394791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:27.394791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:27.394791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:27.394791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:27.394791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:27.394791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:27.395630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:27.395630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:27.395630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:27.395630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:27.395630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:27.395630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:27.395630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:27.395630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:27.395630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:27.395630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:27.395630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 15:12:27.395630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:27.396257 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 228/438] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:27.401338 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08e627-21c2-4616-abc7-1bc210f3e1aa None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:27.472698 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:27.520891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc55de9-ac81-429a-857b-7c575287d4ab None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:27.528617 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08e627-21c2-4616-abc7-1bc210f3e1aa None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:27.555709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:27.576066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de496447-a694-49d4-bec6-c0174d88489e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:27.576066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de496447-a694-49d4-bec6-c0174d88489e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:27.576066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de496447-a694-49d4-bec6-c0174d88489e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:27.576365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc614e57-ddac-4a5d-ae68-3d656446aa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:27.576471 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de496447-a694-49d4-bec6-c0174d88489e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:27.576544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc614e57-ddac-4a5d-ae68-3d656446aa1e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:27.576544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc614e57-ddac-4a5d-ae68-3d656446aa1e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:27.576544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc614e57-ddac-4a5d-ae68-3d656446aa1e None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:27.576544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc614e57-ddac-4a5d-ae68-3d656446aa1e None admin] RBAC: Adding query filter params (name=member) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:27.576544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc614e57-ddac-4a5d-ae68-3d656446aa1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:27.576829 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc614e57-ddac-4a5d-ae68-3d656446aa1e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:27.586403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 229/439] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:12:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:12:27.591928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc614e57-ddac-4a5d-ae68-3d656446aa1e None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:27.670219 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de496447-a694-49d4-bec6-c0174d88489e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:27.671759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de496447-a694-49d4-bec6-c0174d88489e None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:27.721850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc614e57-ddac-4a5d-ae68-3d656446aa1e None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:27.763725 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a96a4-4606-4933-a88c-8639ebe3d9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:27.764394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a96a4-4606-4933-a88c-8639ebe3d9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:27.764681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a96a4-4606-4933-a88c-8639ebe3d9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:27.764963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a96a4-4606-4933-a88c-8639ebe3d9e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:27.766361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:27.806330 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 211/440] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 15:12:27.808813 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89e4f4-b4a1-4e12-bfcf-5a3a9a0bf892 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:27.809237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89e4f4-b4a1-4e12-bfcf-5a3a9a0bf892 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:27.809802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89e4f4-b4a1-4e12-bfcf-5a3a9a0bf892 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:27.811459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 212/441] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:27.812345 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89e4f4-b4a1-4e12-bfcf-5a3a9a0bf892 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:27.833647 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:27.851409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a96a4-4606-4933-a88c-8639ebe3d9e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:27.853162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a96a4-4606-4933-a88c-8639ebe3d9e1 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:27.921256 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89e4f4-b4a1-4e12-bfcf-5a3a9a0bf892 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:27.952719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 230/442] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:27.954067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d6604-af7f-4408-b83c-6d68e2c4f787 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:27.954405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d6604-af7f-4408-b83c-6d68e2c4f787 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:27.954660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d6604-af7f-4408-b83c-6d68e2c4f787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:27.959635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d974bd-89f1-4571-b49c-180d9e02dba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:27.960479 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d974bd-89f1-4571-b49c-180d9e02dba9 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:27.960759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d974bd-89f1-4571-b49c-180d9e02dba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:27.961029 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d974bd-89f1-4571-b49c-180d9e02dba9 None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:27.962203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d974bd-89f1-4571-b49c-180d9e02dba9 None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:27.963643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d974bd-89f1-4571-b49c-180d9e02dba9 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:27.969565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 213/443] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:27 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:27.969565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d974bd-89f1-4571-b49c-180d9e02dba9 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:27.972079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-ae1d6604-af7f-4408-b83c-6d68e2c4f787 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:27.973766 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ae1d6604-af7f-4408-b83c-6d68e2c4f787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:28.008442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:28.040614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 231/444] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:28.041660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d6604-af7f-4408-b83c-6d68e2c4f787 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:28.062875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:28.069201 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d974bd-89f1-4571-b49c-180d9e02dba9 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:28.105498 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e4fe6-f4d0-4c17-a462-ff9e4b5bd160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:28.106448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e4fe6-f4d0-4c17-a462-ff9e4b5bd160 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:28.106876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e4fe6-f4d0-4c17-a462-ff9e4b5bd160 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:28.107152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e4fe6-f4d0-4c17-a462-ff9e4b5bd160 None admin] PATH_INFO: `/v3/projects/b271a003871a42fb99fa8b9ca8f9017f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:28.108346 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e4fe6-f4d0-4c17-a462-ff9e4b5bd160 None admin] RBAC: Authorizing `identity:get_project(project_id=b271a003871a42fb99fa8b9ca8f9017f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:28.109759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e4fe6-f4d0-4c17-a462-ff9e4b5bd160 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:28.111453 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 214/445] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:27 2022] GET /identity/v3/projects/b271a003871a42fb99fa8b9ca8f9017f => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:28.112207 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e4fe6-f4d0-4c17-a462-ff9e4b5bd160 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:28.151789 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d6604-af7f-4408-b83c-6d68e2c4f787 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:28.184338 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4acadb-41fe-44fd-ba24-6d8968f4c671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:28.185327 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4acadb-41fe-44fd-ba24-6d8968f4c671 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:28.185605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4acadb-41fe-44fd-ba24-6d8968f4c671 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:28.185877 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4acadb-41fe-44fd-ba24-6d8968f4c671 None admin] PATH_INFO: `/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:28.187073 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4acadb-41fe-44fd-ba24-6d8968f4c671 None admin] RBAC: Authorizing `identity:get_group(group_id=921a4e3d2fdf49ba82cc42dc6f7f9737)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:28.188525 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4acadb-41fe-44fd-ba24-6d8968f4c671 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:28.190120 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 232/446] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Jul 20 15:12:28 2022] GET /identity/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:28.190917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4acadb-41fe-44fd-ba24-6d8968f4c671 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:28.211084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e4fe6-f4d0-4c17-a462-ff9e4b5bd160 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:28.220817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:28.246247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea86de-9f27-4aad-98af-9d98ca1af59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:28.247070 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea86de-9f27-4aad-98af-9d98ca1af59f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:28.247357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea86de-9f27-4aad-98af-9d98ca1af59f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:28.247649 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea86de-9f27-4aad-98af-9d98ca1af59f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:28.258604 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea86de-9f27-4aad-98af-9d98ca1af59f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:28.266216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-7fea86de-9f27-4aad-98af-9d98ca1af59f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:12:28.266216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:28.266216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:28.266216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:28.266216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:28.266216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:28.266216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:28.266216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:28.266216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:28.266216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:28.266216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:28.266216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:28.266216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:28.266216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:28.266216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:28.266216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:28.267168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:28.267168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:28.267168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:28.267168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:28.267168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:28.267168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:28.267168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:28.267168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:28.267168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:28.267168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:28.267168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:28.267168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:28.267168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:28.267168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:28.267168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:28.267168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:28.267984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:28.267984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:28.267984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:28.267984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:28.267984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:28.267984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:28.267984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:28.267984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:28.267984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:28.267984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:28.267984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:12:28.267984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:28.267984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 215/447] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:12:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:28.268746 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea86de-9f27-4aad-98af-9d98ca1af59f None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:28.282433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:28.297826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4acadb-41fe-44fd-ba24-6d8968f4c671 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:28.331233 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215d44d-f37c-4c87-a12c-972da0b7d175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:28.332100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215d44d-f37c-4c87-a12c-972da0b7d175 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:28.332387 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215d44d-f37c-4c87-a12c-972da0b7d175 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:28.333179 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215d44d-f37c-4c87-a12c-972da0b7d175 None admin] PATH_INFO: `/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:28.334394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215d44d-f37c-4c87-a12c-972da0b7d175 None admin] RBAC: Authorizing `identity:get_group(group_id=921a4e3d2fdf49ba82cc42dc6f7f9737)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:28.335843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215d44d-f37c-4c87-a12c-972da0b7d175 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:28.337773 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 233/448] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Jul 20 15:12:28 2022] GET /identity/v3/groups/921a4e3d2fdf49ba82cc42dc6f7f9737 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:28.338356 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215d44d-f37c-4c87-a12c-972da0b7d175 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:28.373481 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea86de-9f27-4aad-98af-9d98ca1af59f None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:28.406879 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee3876-8de4-43d2-9b6b-e628e5551c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:28.407528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee3876-8de4-43d2-9b6b-e628e5551c60 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:28.407816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee3876-8de4-43d2-9b6b-e628e5551c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:28.408098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee3876-8de4-43d2-9b6b-e628e5551c60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:28.408936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee3876-8de4-43d2-9b6b-e628e5551c60 None admin] RBAC: Adding query filter params (group.id=921a4e3d2fdf49ba82cc42dc6f7f9737, role.id=2da8d544efe44ce6a41160d5759be402, scope.project.id=b271a003871a42fb99fa8b9ca8f9017f) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:28.409248 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee3876-8de4-43d2-9b6b-e628e5551c60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:28.410723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee3876-8de4-43d2-9b6b-e628e5551c60 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:28.416830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 216/449] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Jul 20 15:12:28 2022] GET /identity/v3/role_assignments?group.id=921a4e3d2fdf49ba82cc42dc6f7f9737&scope.project.id=b271a003871a42fb99fa8b9ca8f9017f&role.id=2da8d544efe44ce6a41160d5759be402 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:28.417606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee3876-8de4-43d2-9b6b-e628e5551c60 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:28.440856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215d44d-f37c-4c87-a12c-972da0b7d175 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:28.455479 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:28.482669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b55eeb-b380-4406-be09-7ea65216e98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:28.483666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b55eeb-b380-4406-be09-7ea65216e98c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:28.484247 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b55eeb-b380-4406-be09-7ea65216e98c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:28.485056 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b55eeb-b380-4406-be09-7ea65216e98c None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:28.486303 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b55eeb-b380-4406-be09-7ea65216e98c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:28.486770 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b55eeb-b380-4406-be09-7ea65216e98c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:28.489128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b55eeb-b380-4406-be09-7ea65216e98c None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:28.501841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 234/450] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:28.508462 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b55eeb-b380-4406-be09-7ea65216e98c None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:28.528634 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:28.532073 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee3876-8de4-43d2-9b6b-e628e5551c60 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:28.572302 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c4a07-99e7-4ba3-bc48-da0b3eeb3888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:28.573353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c4a07-99e7-4ba3-bc48-da0b3eeb3888 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:28.573353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c4a07-99e7-4ba3-bc48-da0b3eeb3888 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:28.573522 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c4a07-99e7-4ba3-bc48-da0b3eeb3888 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:28.574382 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c4a07-99e7-4ba3-bc48-da0b3eeb3888 None admin] RBAC: Adding query filter params (group.id=921a4e3d2fdf49ba82cc42dc6f7f9737, role.id=d85783606f6649f5b9465f407cafb610, scope.project.id=b271a003871a42fb99fa8b9ca8f9017f) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:28.574704 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c4a07-99e7-4ba3-bc48-da0b3eeb3888 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:28.576580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c4a07-99e7-4ba3-bc48-da0b3eeb3888 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:28.583409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 217/451] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Jul 20 15:12:28 2022] GET /identity/v3/role_assignments?group.id=921a4e3d2fdf49ba82cc42dc6f7f9737&scope.project.id=b271a003871a42fb99fa8b9ca8f9017f&role.id=d85783606f6649f5b9465f407cafb610 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:28.634905 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b55eeb-b380-4406-be09-7ea65216e98c None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:28.674295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216dd80d-6ea3-46ad-931b-ac6d682b9e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:28.674955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dd80d-6ea3-46ad-931b-ac6d682b9e63 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:28.675241 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dd80d-6ea3-46ad-931b-ac6d682b9e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:28.675519 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dd80d-6ea3-46ad-931b-ac6d682b9e63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:28.745279 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216dd80d-6ea3-46ad-931b-ac6d682b9e63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:28.746984 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216dd80d-6ea3-46ad-931b-ac6d682b9e63 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:28.761676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:28.846587 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 235/452] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:28.852792 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c4a07-99e7-4ba3-bc48-da0b3eeb3888 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:28.950316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:28.955926 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c4a07-99e7-4ba3-bc48-da0b3eeb3888 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:28.989975 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e81712-deba-4e41-932d-464635906309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:28.990864 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e81712-deba-4e41-932d-464635906309 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:28.991143 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e81712-deba-4e41-932d-464635906309 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:28.991415 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e81712-deba-4e41-932d-464635906309 None admin] PATH_INFO: `/v3/projects/effa579779b9494f8b1dd5831edaae25` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:28.992589 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e81712-deba-4e41-932d-464635906309 None admin] RBAC: Authorizing `identity:get_project(project_id=effa579779b9494f8b1dd5831edaae25)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:28.994054 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e81712-deba-4e41-932d-464635906309 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:28.995764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 218/453] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:12:28 2022] GET /identity/v3/projects/effa579779b9494f8b1dd5831edaae25 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:29.001431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216dd80d-6ea3-46ad-931b-ac6d682b9e63 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:29.102216 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216dd80d-6ea3-46ad-931b-ac6d682b9e63 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:29.115550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:29.135399 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476af94f-e74c-426f-879f-febf7c5afe2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:29.136247 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476af94f-e74c-426f-879f-febf7c5afe2f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:29.136528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476af94f-e74c-426f-879f-febf7c5afe2f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:29.136799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476af94f-e74c-426f-879f-febf7c5afe2f None admin] PATH_INFO: `/v3/groups/2e08bdaa12454430a3462f84c00cb89f` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:29.137960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476af94f-e74c-426f-879f-febf7c5afe2f None admin] RBAC: Authorizing `identity:get_group(group_id=2e08bdaa12454430a3462f84c00cb89f)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:29.139428 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476af94f-e74c-426f-879f-febf7c5afe2f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:29.141027 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 236/454] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Jul 20 15:12:29 2022] GET /identity/v3/groups/2e08bdaa12454430a3462f84c00cb89f => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:29.147561 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e81712-deba-4e41-932d-464635906309 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:29.243338 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e81712-deba-4e41-932d-464635906309 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:29.276378 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e8b8d-089d-48b2-b91e-c6c70c45e5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:29.277019 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e8b8d-089d-48b2-b91e-c6c70c45e5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:29.277304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e8b8d-089d-48b2-b91e-c6c70c45e5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:29.277580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e8b8d-089d-48b2-b91e-c6c70c45e5e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:29.278426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e8b8d-089d-48b2-b91e-c6c70c45e5e7 None admin] RBAC: Adding query filter params (group.id=2e08bdaa12454430a3462f84c00cb89f, role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=effa579779b9494f8b1dd5831edaae25) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:29.278741 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e8b8d-089d-48b2-b91e-c6c70c45e5e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:29.280156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e8b8d-089d-48b2-b91e-c6c70c45e5e7 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:29.285831 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 219/455] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Jul 20 15:12:29 2022] GET /identity/v3/role_assignments?group.id=2e08bdaa12454430a3462f84c00cb89f&scope.project.id=effa579779b9494f8b1dd5831edaae25&role.id=25516be1485c49538544ea8a596e7d17 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:31.725622 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abb1df-9647-41fd-97ca-8076378b9570 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:31.726529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abb1df-9647-41fd-97ca-8076378b9570 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:31.726983 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abb1df-9647-41fd-97ca-8076378b9570 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:31.728510 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 237/456] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:31.733870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebd4a7-4fa3-4a72-ac41-06810dbcbd1c None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:31.734644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebd4a7-4fa3-4a72-ac41-06810dbcbd1c None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:31.735076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebd4a7-4fa3-4a72-ac41-06810dbcbd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:31.752556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-43ebd4a7-4fa3-4a72-ac41-06810dbcbd1c None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:31.754216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-43ebd4a7-4fa3-4a72-ac41-06810dbcbd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:31.786757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:31.838954 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 220/457] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:31.884303 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abb1df-9647-41fd-97ca-8076378b9570 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:32.012541 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cec22d-0aeb-4c17-b977-2326b4a8a668 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:32.013363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cec22d-0aeb-4c17-b977-2326b4a8a668 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:32.013838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cec22d-0aeb-4c17-b977-2326b4a8a668 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:32.016703 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abb1df-9647-41fd-97ca-8076378b9570 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:32.016999 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 221/458] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:12:32.021582 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a15337-4dc3-4903-8b34-37f1794ca509 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:32.022125 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a15337-4dc3-4903-8b34-37f1794ca509 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:32.022632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a15337-4dc3-4903-8b34-37f1794ca509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:32.039548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-f6a15337-4dc3-4903-8b34-37f1794ca509 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:32.041051 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f6a15337-4dc3-4903-8b34-37f1794ca509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:32.075176 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08851060-20c4-4dfb-91e2-a2ff0f22dd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:32.076492 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08851060-20c4-4dfb-91e2-a2ff0f22dd06 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:32.076970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08851060-20c4-4dfb-91e2-a2ff0f22dd06 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:32.077438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08851060-20c4-4dfb-91e2-a2ff0f22dd06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:32.088057 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08851060-20c4-4dfb-91e2-a2ff0f22dd06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:32.089802 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08851060-20c4-4dfb-91e2-a2ff0f22dd06 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:32.122972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-08851060-20c4-4dfb-91e2-a2ff0f22dd06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:32.122972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:32.122972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:32.122972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:32.122972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:32.122972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:32.122972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:32.122972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:32.122972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:32.122972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:32.122972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:32.122972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:32.122972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:32.122972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:32.122972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:32.123901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:32.123901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:32.123901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:32.123901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:32.123901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:32.123901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:32.123901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:32.123901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:32.123901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:32.123901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:32.123901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:32.123901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:32.123901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:32.123901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:32.123901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:32.123901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:32.123901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:32.124773 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:32.124773 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:32.124773 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:32.124773 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:32.132463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 238/459] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:32.134038 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc7b17-e9e9-4ca8-93b3-2247c79982f3 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:32.134401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc7b17-e9e9-4ca8-93b3-2247c79982f3 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:32.134855 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc7b17-e9e9-4ca8-93b3-2247c79982f3 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:32.136540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 239/460] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:32.137474 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc7b17-e9e9-4ca8-93b3-2247c79982f3 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:32.146110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 222/461] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:12:32.147672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccd5e7-8636-4ffc-9040-0144cc7235e5 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:32.148091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccd5e7-8636-4ffc-9040-0144cc7235e5 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:32.148494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccd5e7-8636-4ffc-9040-0144cc7235e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:32.168519 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-59ccd5e7-8636-4ffc-9040-0144cc7235e5 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:32.170803 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-59ccd5e7-8636-4ffc-9040-0144cc7235e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:32.197643 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:32.265648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 223/462] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:32.266191 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc7b17-e9e9-4ca8-93b3-2247c79982f3 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:32.271341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccd5e7-8636-4ffc-9040-0144cc7235e5 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:32.308222 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:32.310981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f493e1-465c-4885-8ace-22733288867a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:32.311660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f493e1-465c-4885-8ace-22733288867a None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:32.312395 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f493e1-465c-4885-8ace-22733288867a None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:32.314558 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f493e1-465c-4885-8ace-22733288867a None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:32.314558 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f493e1-465c-4885-8ace-22733288867a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:32.314558 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f493e1-465c-4885-8ace-22733288867a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:32.315730 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f493e1-465c-4885-8ace-22733288867a None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:32.338387 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 240/463] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:32.347327 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f493e1-465c-4885-8ace-22733288867a None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:32.411851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccd5e7-8636-4ffc-9040-0144cc7235e5 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:32.448027 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:32.452322 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a39be-5f7a-44c7-9fc6-e0ec59d8fd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:32.453121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a39be-5f7a-44c7-9fc6-e0ec59d8fd19 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:32.453404 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a39be-5f7a-44c7-9fc6-e0ec59d8fd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:32.453680 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a39be-5f7a-44c7-9fc6-e0ec59d8fd19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:32.456648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f493e1-465c-4885-8ace-22733288867a None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:32.464019 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a39be-5f7a-44c7-9fc6-e0ec59d8fd19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:32.465480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a39be-5f7a-44c7-9fc6-e0ec59d8fd19 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:32.484210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-209a39be-5f7a-44c7-9fc6-e0ec59d8fd19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:32.484210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:32.484210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:32.484210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:32.484210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:32.484210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:32.484210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:32.484210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:32.484210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:32.484210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:32.484210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:32.484210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:32.484210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:32.484210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:32.484210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:32.485257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:32.485257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:32.485257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:32.485257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:32.485257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:32.485257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:32.485257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:32.485257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:32.485257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:32.485257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:32.485257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:32.485257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:32.485257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:32.485257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:32.485257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:32.485257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:32.485257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:32.486307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:32.486307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:32.486307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:32.486307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:32.490634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 224/464] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:32.491444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a39be-5f7a-44c7-9fc6-e0ec59d8fd19 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:32.499146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e44321-4ea2-45d6-b350-ca21edb3cc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:32.500550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e44321-4ea2-45d6-b350-ca21edb3cc64 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:32.500975 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e44321-4ea2-45d6-b350-ca21edb3cc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:32.501390 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e44321-4ea2-45d6-b350-ca21edb3cc64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:32.514185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e44321-4ea2-45d6-b350-ca21edb3cc64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:32.515727 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e44321-4ea2-45d6-b350-ca21edb3cc64 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:32.534275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-f4e44321-4ea2-45d6-b350-ca21edb3cc64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:32.534275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:32.534275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:32.534275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:32.534275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:32.534275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:32.534275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:32.534275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:32.534275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:32.534275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:32.534275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:32.534275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:32.534275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:32.534275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:32.534275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:32.535149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:32.535149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:32.535149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:32.535149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:32.535149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:32.535149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:32.535149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:32.535149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:32.535149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:32.535149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:32.535149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:32.535149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:32.535149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:32.535149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:32.535149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:32.535149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:32.535149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:32.536591 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:32.536591 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:32.536591 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:32.536591 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:32.536591 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 241/465] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:32.537366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb881c67-6567-4452-9b7c-56f453d21c76 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:32.537804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb881c67-6567-4452-9b7c-56f453d21c76 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:32.537903 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb881c67-6567-4452-9b7c-56f453d21c76 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:32.539256 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 242/466] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:32.542434 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb881c67-6567-4452-9b7c-56f453d21c76 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:32.569070 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:32.602398 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a39be-5f7a-44c7-9fc6-e0ec59d8fd19 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:32.637026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465d450-669a-4085-b8d2-4c2bd28511ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:32.637701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465d450-669a-4085-b8d2-4c2bd28511ee None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:32.638014 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465d450-669a-4085-b8d2-4c2bd28511ee None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:32.638287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465d450-669a-4085-b8d2-4c2bd28511ee None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:32.639004 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465d450-669a-4085-b8d2-4c2bd28511ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:32.640454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465d450-669a-4085-b8d2-4c2bd28511ee None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:32.652021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb881c67-6567-4452-9b7c-56f453d21c76 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:32.685490 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b58c6f-cf91-4d1a-bf6c-0deaa4c30877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:32.686147 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b58c6f-cf91-4d1a-bf6c-0deaa4c30877 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:32.686465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b58c6f-cf91-4d1a-bf6c-0deaa4c30877 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:32.686738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b58c6f-cf91-4d1a-bf6c-0deaa4c30877 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:32.687446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b58c6f-cf91-4d1a-bf6c-0deaa4c30877 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:32.687750 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b58c6f-cf91-4d1a-bf6c-0deaa4c30877 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:32.689105 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b58c6f-cf91-4d1a-bf6c-0deaa4c30877 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:32.700294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 243/467] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:32.701079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b58c6f-cf91-4d1a-bf6c-0deaa4c30877 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:32.711908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:32.757950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 225/468] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:12:32 2022] POST /identity/v3/users => generated 246 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:12:32.757950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331761e2-ab7a-468f-95cc-389625017d98 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:32.757950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331761e2-ab7a-468f-95cc-389625017d98 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:32.757950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331761e2-ab7a-468f-95cc-389625017d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:32.787524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-331761e2-ab7a-468f-95cc-389625017d98 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:32.789397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-331761e2-ab7a-468f-95cc-389625017d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:32.818556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b58c6f-cf91-4d1a-bf6c-0deaa4c30877 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:32.868646 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1029bf5-da36-4354-90ac-92ecf4c2a57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:32.869865 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1029bf5-da36-4354-90ac-92ecf4c2a57b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:32.870528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1029bf5-da36-4354-90ac-92ecf4c2a57b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:32.870908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1029bf5-da36-4354-90ac-92ecf4c2a57b None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:32.871909 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1029bf5-da36-4354-90ac-92ecf4c2a57b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:32.872365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1029bf5-da36-4354-90ac-92ecf4c2a57b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:32.874476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1029bf5-da36-4354-90ac-92ecf4c2a57b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:32.883466 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 226/469] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:32.884441 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331761e2-ab7a-468f-95cc-389625017d98 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:32.889303 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 244/470] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:32.894738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1029bf5-da36-4354-90ac-92ecf4c2a57b None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:32.972437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:32.978797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:33.026449 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1029bf5-da36-4354-90ac-92ecf4c2a57b None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:33.028125 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331761e2-ab7a-468f-95cc-389625017d98 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:33.064821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e186380-45c1-42af-8d2e-2e6a5250084a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:33.064821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e186380-45c1-42af-8d2e-2e6a5250084a None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:33.064821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e186380-45c1-42af-8d2e-2e6a5250084a None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:33.065145 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e186380-45c1-42af-8d2e-2e6a5250084a None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:33.066066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e186380-45c1-42af-8d2e-2e6a5250084a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:33.068547 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e186380-45c1-42af-8d2e-2e6a5250084a None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:33.073231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444bff13-8334-4688-9974-326859e323ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:33.074139 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444bff13-8334-4688-9974-326859e323ae None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:33.074659 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444bff13-8334-4688-9974-326859e323ae None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:33.075141 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444bff13-8334-4688-9974-326859e323ae None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:33.076217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444bff13-8334-4688-9974-326859e323ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:33.079008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444bff13-8334-4688-9974-326859e323ae None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:33.166347 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 245/471] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:12:32 2022] POST /identity/v3/users => generated 248 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:12:33.166347 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e186380-45c1-42af-8d2e-2e6a5250084a None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:33.194580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 227/472] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:12:32 2022] POST /identity/v3/users => generated 248 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:12:33.296335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e186380-45c1-42af-8d2e-2e6a5250084a None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:33.325736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:33.339921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418546bd-df0e-41e5-9127-b5517ecea0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:33.341092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418546bd-df0e-41e5-9127-b5517ecea0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:33.341579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418546bd-df0e-41e5-9127-b5517ecea0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:33.342084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418546bd-df0e-41e5-9127-b5517ecea0a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:33.354607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418546bd-df0e-41e5-9127-b5517ecea0a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:33.356422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418546bd-df0e-41e5-9127-b5517ecea0a1 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:33.376821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-418546bd-df0e-41e5-9127-b5517ecea0a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:33.376821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:33.376821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:33.376821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:33.376821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:33.376821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:33.376821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:33.376821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:33.376821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:33.376821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:33.376821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:33.376821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:33.376821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:33.376821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:33.376821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:33.378518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:33.378518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:33.378518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:33.378518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:33.378518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:33.378518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:33.378518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:33.378518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:33.378518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:33.378518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:33.378518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:33.378518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:33.378518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:33.378518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:33.378518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:33.378518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:33.378518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:33.379959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:33.379959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:33.379959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:33.379959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:33.381631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 246/473] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:33.388390 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444bff13-8334-4688-9974-326859e323ae None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:33.409710 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd081f9-648f-4ee9-9b33-f97124f5f00e None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:33.410381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd081f9-648f-4ee9-9b33-f97124f5f00e None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:33.410950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd081f9-648f-4ee9-9b33-f97124f5f00e None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:33.413342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 247/474] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:12:33.422732 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f5891-f052-4f57-9549-7619ea7fe50a None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:33.423341 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f5891-f052-4f57-9549-7619ea7fe50a None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:33.423826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f5891-f052-4f57-9549-7619ea7fe50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:33.446840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-fb3f5891-f052-4f57-9549-7619ea7fe50a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:33.448527 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-fb3f5891-f052-4f57-9549-7619ea7fe50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:33.526102 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444bff13-8334-4688-9974-326859e323ae None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:33.532033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 248/475] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:12:33.540284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:33.561588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe9387-92db-4784-9e88-e93df6939175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:33.562640 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe9387-92db-4784-9e88-e93df6939175 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:33.562929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe9387-92db-4784-9e88-e93df6939175 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:33.563048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f5891-f052-4f57-9549-7619ea7fe50a None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:33.563208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe9387-92db-4784-9e88-e93df6939175 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:33.563931 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe9387-92db-4784-9e88-e93df6939175 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:33.564237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe9387-92db-4784-9e88-e93df6939175 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:33.565726 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe9387-92db-4784-9e88-e93df6939175 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:33.580116 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 228/476] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:33.586485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe9387-92db-4784-9e88-e93df6939175 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:33.708731 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe9387-92db-4784-9e88-e93df6939175 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:33.751238 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991fc0f-fc13-49b8-954a-e7b94e2570c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:33.752277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991fc0f-fc13-49b8-954a-e7b94e2570c3 None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:33.752765 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991fc0f-fc13-49b8-954a-e7b94e2570c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:33.753498 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991fc0f-fc13-49b8-954a-e7b94e2570c3 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:33.754468 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991fc0f-fc13-49b8-954a-e7b94e2570c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:33.756764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991fc0f-fc13-49b8-954a-e7b94e2570c3 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:33.774185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f5891-f052-4f57-9549-7619ea7fe50a None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:33.777027 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:33.810969 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6518d-4111-430d-af50-960540e6976f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:33.812058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6518d-4111-430d-af50-960540e6976f None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:33.812570 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6518d-4111-430d-af50-960540e6976f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:33.813072 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6518d-4111-430d-af50-960540e6976f None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:33.814209 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6518d-4111-430d-af50-960540e6976f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:33.815923 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6518d-4111-430d-af50-960540e6976f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:33.853262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 229/477] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:12:33 2022] POST /identity/v3/users => generated 249 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:12:33.880891 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.sql.core [None req-f5f6518d-4111-430d-af50-960540e6976f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 15:12:33.880891 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 15:12:33.880891 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [parameters: {'id': 'e0ca8e82001940459633e03dfd66db7c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 15:12:33.880891 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80508) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 15:12:33.904681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-f5f6518d-4111-430d-af50-960540e6976f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 15:12:33.904681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:33.904681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 15:12:33.904681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 15:12:33.904681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 15:12:33.904681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 15:12:33.904681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 15:12:33.904681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 15:12:33.904681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 15:12:33.904681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application conn.query(q) Jul 20 15:12:33.904681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 15:12:33.904681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 15:12:33.904681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 15:12:33.904681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application result.read() Jul 20 15:12:33.904681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return ref.to_dict() Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 15:12:33.905638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application next(self.gen) Jul 20 15:12:33.906711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 15:12:33.906711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application yield resource Jul 20 15:12:33.906711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 15:12:33.906711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application next(self.gen) Jul 20 15:12:33.906711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 15:12:33.906711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 15:12:33.906711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 15:12:33.906711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 15:12:33.906711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 15:12:33.906711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise self.value Jul 20 15:12:33.906711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 15:12:33.906711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 15:12:33.906711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 15:12:33.906711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application session.commit() Jul 20 15:12:33.906711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 15:12:33.906711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 15:12:33.906711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 15:12:33.907627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 15:12:33.907627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 15:12:33.907627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self.session.flush() Jul 20 15:12:33.907627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 15:12:33.907627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 15:12:33.907627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 15:12:33.907627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 15:12:33.907627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 15:12:33.907627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application compat.raise_( Jul 20 15:12:33.907627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 15:12:33.907627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception Jul 20 15:12:33.907627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 15:12:33.907627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 15:12:33.907627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 15:12:33.907627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 15:12:33.907627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 15:12:33.907627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 15:12:33.908544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 15:12:33.908544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 15:12:33.908544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 15:12:33.908544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 15:12:33.908544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 15:12:33.908544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 15:12:33.908544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 15:12:33.908544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 15:12:33.908544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 15:12:33.908544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 15:12:33.908544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 15:12:33.908544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 15:12:33.908544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 15:12:33.908544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 15:12:33.908544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 15:12:33.908544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception Jul 20 15:12:33.909382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 15:12:33.909382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 15:12:33.909382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 15:12:33.909382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 15:12:33.909382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 15:12:33.909382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 15:12:33.909382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 15:12:33.909382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application conn.query(q) Jul 20 15:12:33.909382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 15:12:33.909382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 15:12:33.909382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 15:12:33.909382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application result.read() Jul 20 15:12:33.909382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 15:12:33.909382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 15:12:33.909382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 15:12:33.909382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 15:12:33.910189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 15:12:33.910189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 15:12:33.910189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 15:12:33.910189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 15:12:33.910189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 15:12:33.910189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 15:12:33.910189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [parameters: {'id': 'e0ca8e82001940459633e03dfd66db7c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 15:12:33.910189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 15:12:33.910189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:33.910189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:12:33.910189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:33.910189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:33.910189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:33.910189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:33.910189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:33.910189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:33.917384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:33.917384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:33.917384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:33.917384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:33.917384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:33.917384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:33.917384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 20 15:12:33.917384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 20 15:12:33.917384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:33.917384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:33.917384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 20 15:12:33.917384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 20 15:12:33.917384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 15:12:33.917384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 15:12:33.917384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 15:12:33.917384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:33.921041 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 249/478] 10.222.0.78 () {60 vars in 1131 bytes} [Wed Jul 20 15:12:33 2022] POST /identity/v3/roles => generated 144 bytes in 359 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:33.927209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991fc0f-fc13-49b8-954a-e7b94e2570c3 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:33.974675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:34.078643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991fc0f-fc13-49b8-954a-e7b94e2570c3 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:34.118968 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b856fe-ebf9-4133-b63a-92515fd1ede9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:34.119709 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b856fe-ebf9-4133-b63a-92515fd1ede9 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:34.119987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b856fe-ebf9-4133-b63a-92515fd1ede9 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:34.120261 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b856fe-ebf9-4133-b63a-92515fd1ede9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:34.132097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b856fe-ebf9-4133-b63a-92515fd1ede9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:34.140212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-c7b856fe-ebf9-4133-b63a-92515fd1ede9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:34.140212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:34.140212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:34.140212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:34.140212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:34.140212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:34.140212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:34.140212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:34.140212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:34.140212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:34.140212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:34.140212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:34.140212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:34.140212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:34.140212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:34.140212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:34.141288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:34.141288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:34.141288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:34.141288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:34.141288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:34.141288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:34.141288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:34.141288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:34.141288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:34.141288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:34.141288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:34.141288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:34.141288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:34.141288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:34.141288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:34.141288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:34.142335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:34.142335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:34.142335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:34.142335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:34.142335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:34.142335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:34.142335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:34.142335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:34.142335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:34.142335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:34.142335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 15:12:34.142335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:34.142335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 230/479] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:12:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:34.148234 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6518d-4111-430d-af50-960540e6976f None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:34.255162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6518d-4111-430d-af50-960540e6976f None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:34.292302 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbee32-1218-4797-aa70-66fe8470ea81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:34.293233 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbee32-1218-4797-aa70-66fe8470ea81 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:34.293556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbee32-1218-4797-aa70-66fe8470ea81 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:34.293833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbee32-1218-4797-aa70-66fe8470ea81 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:34.294613 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dbee32-1218-4797-aa70-66fe8470ea81 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:34.294918 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dbee32-1218-4797-aa70-66fe8470ea81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:34.296283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dbee32-1218-4797-aa70-66fe8470ea81 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:34.305436 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 250/480] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:35.086384 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6923d-8fde-4553-93af-7272eea8ebb0 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:35.086926 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6923d-8fde-4553-93af-7272eea8ebb0 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:35.086926 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6923d-8fde-4553-93af-7272eea8ebb0 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:35.088565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 231/481] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:35.094165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a1664-e06c-4d53-81d1-ccbb0d6e3b49 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:35.094619 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a1664-e06c-4d53-81d1-ccbb0d6e3b49 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:35.094792 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a1664-e06c-4d53-81d1-ccbb0d6e3b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:35.121226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-fc0a1664-e06c-4d53-81d1-ccbb0d6e3b49 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:35.122470 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-fc0a1664-e06c-4d53-81d1-ccbb0d6e3b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:35.149175 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:35.194869 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 251/482] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:12:35.218935 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6923d-8fde-4553-93af-7272eea8ebb0 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:35.309970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:35.331684 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6923d-8fde-4553-93af-7272eea8ebb0 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:35.380789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ccb577-7573-4b0b-964c-af5c3ba6b916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:35.381518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccb577-7573-4b0b-964c-af5c3ba6b916 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:35.381804 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccb577-7573-4b0b-964c-af5c3ba6b916 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:35.382081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccb577-7573-4b0b-964c-af5c3ba6b916 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:35.393175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ccb577-7573-4b0b-964c-af5c3ba6b916 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:35.401185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-d6ccb577-7573-4b0b-964c-af5c3ba6b916 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:35.401185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:35.401185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:35.401185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:35.401185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:35.401185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:35.401185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:35.401185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:35.401185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:35.401185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:35.401185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:35.401185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:35.401185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:35.401185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:35.401185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:35.401185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:35.402112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:35.402112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:35.402112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:35.402112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:35.402112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:35.402112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:35.402112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:35.402112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:35.402112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:35.402112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:35.402112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:35.402112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:35.402112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:35.402112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:35.402112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:35.402112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:35.403157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:35.403157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:35.403157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:35.403157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:35.403157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:35.403157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:35.403157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:35.403157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:35.403157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:35.403157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:35.403157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:35.403157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:35.403157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 232/483] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:35.409205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a1664-e06c-4d53-81d1-ccbb0d6e3b49 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:35.552218 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a1664-e06c-4d53-81d1-ccbb0d6e3b49 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:35.588411 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea4f06-6127-4540-aa34-1fa910d5be28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:35.589084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea4f06-6127-4540-aa34-1fa910d5be28 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:35.589361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea4f06-6127-4540-aa34-1fa910d5be28 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:35.589633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea4f06-6127-4540-aa34-1fa910d5be28 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:35.590395 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea4f06-6127-4540-aa34-1fa910d5be28 None admin] RBAC: Adding query filter params (name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:35.590872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea4f06-6127-4540-aa34-1fa910d5be28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:35.592260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea4f06-6127-4540-aa34-1fa910d5be28 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:35.601593 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 252/484] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:12:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:12:35.607019 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ccb577-7573-4b0b-964c-af5c3ba6b916 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:35.675961 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f470b1-ee67-461b-a87b-2354f30eba62 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:35.676226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f470b1-ee67-461b-a87b-2354f30eba62 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:35.676509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f470b1-ee67-461b-a87b-2354f30eba62 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:35.678123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 253/485] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:35.684248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92097249-8217-4354-9d24-d509dd3e3d2b None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:35.684585 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92097249-8217-4354-9d24-d509dd3e3d2b None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:35.684874 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92097249-8217-4354-9d24-d509dd3e3d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:35.716731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-92097249-8217-4354-9d24-d509dd3e3d2b None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:35.718179 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-92097249-8217-4354-9d24-d509dd3e3d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:35.754729 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:35.764448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ccb577-7573-4b0b-964c-af5c3ba6b916 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:35.800126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8be80-eda1-4cfa-904f-5f30e429ef46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:35.801465 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8be80-eda1-4cfa-904f-5f30e429ef46 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:35.801923 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8be80-eda1-4cfa-904f-5f30e429ef46 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:35.802376 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8be80-eda1-4cfa-904f-5f30e429ef46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:35.813632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8be80-eda1-4cfa-904f-5f30e429ef46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:35.815454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8be80-eda1-4cfa-904f-5f30e429ef46 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:35.838672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:35.844110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-4db8be80-eda1-4cfa-904f-5f30e429ef46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:35.844110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:35.844110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:35.844110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:35.844110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:35.844110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:35.844110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:35.844110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:35.844110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:35.844110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:35.844110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:35.844110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:35.844110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:35.844110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:35.844110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:35.844971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:35.844971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:35.844971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:35.844971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:35.844971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:35.844971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:35.844971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:35.844971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:35.844971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:35.844971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:35.844971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:35.844971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:35.844971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:35.844971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:35.844971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:35.844971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:35.844971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:35.845834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:35.845834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:35.845834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:35.845834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:35.846555 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 233/486] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:35.854758 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8be80-eda1-4cfa-904f-5f30e429ef46 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:35.894241 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 254/487] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:35.926958 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92097249-8217-4354-9d24-d509dd3e3d2b None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:35.995080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8be80-eda1-4cfa-904f-5f30e429ef46 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:36.032023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79918d-0745-47a7-b168-03853303d071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:36.033410 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92097249-8217-4354-9d24-d509dd3e3d2b None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:36.033794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79918d-0745-47a7-b168-03853303d071 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:36.034478 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79918d-0745-47a7-b168-03853303d071 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:36.035125 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79918d-0745-47a7-b168-03853303d071 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:36.037154 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79918d-0745-47a7-b168-03853303d071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:36.037772 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79918d-0745-47a7-b168-03853303d071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:36.040344 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79918d-0745-47a7-b168-03853303d071 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:36.067713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 234/488] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:12:36.067713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e46275-29b3-4362-bba6-4d215a44b843 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:36.067713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e46275-29b3-4362-bba6-4d215a44b843 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:36.067713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e46275-29b3-4362-bba6-4d215a44b843 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:36.067713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 235/489] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:36.072378 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e46275-29b3-4362-bba6-4d215a44b843 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:36.081132 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb90b5-c7f6-4639-be0a-0a7a4ab29392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:36.082285 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb90b5-c7f6-4639-be0a-0a7a4ab29392 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:36.083294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb90b5-c7f6-4639-be0a-0a7a4ab29392 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:36.084226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb90b5-c7f6-4639-be0a-0a7a4ab29392 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:36.107618 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb90b5-c7f6-4639-be0a-0a7a4ab29392 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:36.115482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-e5bb90b5-c7f6-4639-be0a-0a7a4ab29392 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:36.115482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:36.115482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:36.115482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:36.115482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:36.115482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:36.115482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:36.115482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:36.115482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:36.115482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:36.115482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:36.115482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:36.115482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:36.115482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:36.115482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:36.115482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:36.116417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:36.116417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:36.116417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:36.116417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:36.116417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:36.116417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:36.116417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:36.116417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:36.116417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:36.116417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:36.116417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:36.116417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:36.116417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:36.116417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:36.116417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:36.116417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:36.117873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:36.117873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:36.117873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:36.117873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:36.117873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:36.117873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:36.117873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:36.117873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:36.117873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:36.117873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:36.117873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:36.117873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:36.117873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 255/490] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:36.118602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3f959-8416-4893-8e0d-f3ec339cf396 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:36.118602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3f959-8416-4893-8e0d-f3ec339cf396 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:36.118746 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3f959-8416-4893-8e0d-f3ec339cf396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:36.136285 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-54c3f959-8416-4893-8e0d-f3ec339cf396 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:36.137718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-54c3f959-8416-4893-8e0d-f3ec339cf396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:36.147304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:36.162492 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:36.210591 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 256/491] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:36.211592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3f959-8416-4893-8e0d-f3ec339cf396 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:36.220075 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e46275-29b3-4362-bba6-4d215a44b843 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:36.259982 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e79dc8-bc18-4388-aa5e-ac054ded0a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:36.261264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e79dc8-bc18-4388-aa5e-ac054ded0a38 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:36.261979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e79dc8-bc18-4388-aa5e-ac054ded0a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:36.262771 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e79dc8-bc18-4388-aa5e-ac054ded0a38 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:36.273759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e79dc8-bc18-4388-aa5e-ac054ded0a38 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:36.275472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e79dc8-bc18-4388-aa5e-ac054ded0a38 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:36.297080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-30e79dc8-bc18-4388-aa5e-ac054ded0a38 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:12:36.297080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:36.297080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:36.297080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:36.297080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:36.297080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:36.297080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:36.297080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:36.297080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:36.297080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:36.297080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:36.297080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:36.297080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:12:36.297080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:12:36.297080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:12:36.297080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:36.298026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:36.299052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:36.299052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:36.299052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:12:36.299052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:12:36.299052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:12:36.299052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:12:36.299052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:12:36.299052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:12:36.299052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:12:36.299052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:36.305369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 236/492] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Jul 20 15:12:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:36.305369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e79dc8-bc18-4388-aa5e-ac054ded0a38 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:36.416922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3f959-8416-4893-8e0d-f3ec339cf396 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:36.448068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e79dc8-bc18-4388-aa5e-ac054ded0a38 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:36.460904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f80ca-f9d4-4797-89b8-45bd7338ba22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:36.461525 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f80ca-f9d4-4797-89b8-45bd7338ba22 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:36.461814 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f80ca-f9d4-4797-89b8-45bd7338ba22 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:36.462097 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f80ca-f9d4-4797-89b8-45bd7338ba22 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:36.462884 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f80ca-f9d4-4797-89b8-45bd7338ba22 None admin] RBAC: Adding query filter params (name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:36.463458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f80ca-f9d4-4797-89b8-45bd7338ba22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:36.465338 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f80ca-f9d4-4797-89b8-45bd7338ba22 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:36.471723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:36.477483 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 257/493] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:12:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:36.478601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f80ca-f9d4-4797-89b8-45bd7338ba22 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:36.483900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:36.488030 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a901643-664f-4bc3-ac40-e1864f1279ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:36.489704 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a901643-664f-4bc3-ac40-e1864f1279ac None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:36.490221 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a901643-664f-4bc3-ac40-e1864f1279ac None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:36.490740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a901643-664f-4bc3-ac40-e1864f1279ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:36.502373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a901643-664f-4bc3-ac40-e1864f1279ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:36.511987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-9a901643-664f-4bc3-ac40-e1864f1279ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:36.511987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:36.511987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:36.511987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:36.511987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:36.511987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:36.511987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:36.511987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:36.511987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:36.511987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:36.511987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:36.511987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:36.511987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:36.511987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:36.511987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:36.511987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:36.512894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:36.512894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:36.512894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:36.512894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:36.512894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:36.512894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:36.512894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:36.512894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:36.512894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:36.512894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:36.512894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:36.512894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:36.512894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:36.512894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:36.512894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:36.512894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:36.513715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:36.513715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:36.513715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:36.513715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:36.513715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:36.513715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:36.513715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:36.513715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:36.513715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:36.513715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:36.513715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:36.513715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:36.516576 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 237/494] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:36.517671 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a901643-664f-4bc3-ac40-e1864f1279ac None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:36.616363 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f80ca-f9d4-4797-89b8-45bd7338ba22 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:36.635916 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a901643-664f-4bc3-ac40-e1864f1279ac None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:36.662579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d2837-c8f0-48d7-9148-95908bb7f347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:36.664240 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d2837-c8f0-48d7-9148-95908bb7f347 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:36.664748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d2837-c8f0-48d7-9148-95908bb7f347 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:36.665463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d2837-c8f0-48d7-9148-95908bb7f347 None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:36.667184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d2837-c8f0-48d7-9148-95908bb7f347 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:36.667724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d2837-c8f0-48d7-9148-95908bb7f347 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:36.669374 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d2837-c8f0-48d7-9148-95908bb7f347 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:36.670740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e311d3d-51f1-4f82-a7bf-1f98dff01052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:36.671536 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e311d3d-51f1-4f82-a7bf-1f98dff01052 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:36.671822 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e311d3d-51f1-4f82-a7bf-1f98dff01052 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:36.672101 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e311d3d-51f1-4f82-a7bf-1f98dff01052 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:36.683400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e311d3d-51f1-4f82-a7bf-1f98dff01052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:36.684859 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e311d3d-51f1-4f82-a7bf-1f98dff01052 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:36.703938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-8e311d3d-51f1-4f82-a7bf-1f98dff01052 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:36.703938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:36.703938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:36.703938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:36.703938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:36.703938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:36.703938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:36.703938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:36.703938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:36.703938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:36.703938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:36.703938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:36.703938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:36.703938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:36.703938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:36.704838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:36.704838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:36.704838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:36.704838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:36.704838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:36.704838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:36.704838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:36.704838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:36.704838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:36.704838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:36.704838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:36.704838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:36.704838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:36.704838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:36.704838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:36.704838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:36.704838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:36.705711 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:36.705711 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:36.705711 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:36.705711 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:36.709744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 238/495] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:36.710475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e311d3d-51f1-4f82-a7bf-1f98dff01052 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:36.716557 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 258/496] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Jul 20 15:12:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:36.716557 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1774d8-36a3-4b37-ae92-34b9461ba3d9 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:36.716557 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1774d8-36a3-4b37-ae92-34b9461ba3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:36.716557 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1774d8-36a3-4b37-ae92-34b9461ba3d9 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:36.716557 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 259/497] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:36.716557 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1774d8-36a3-4b37-ae92-34b9461ba3d9 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:36.749801 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:36.784303 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:36.820322 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e311d3d-51f1-4f82-a7bf-1f98dff01052 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:36.823753 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1774d8-36a3-4b37-ae92-34b9461ba3d9 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:36.858237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c6dd1-9b0f-448e-aade-d1318e520e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:36.858448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae969b47-1bbd-45a1-870a-c08b8cf6ccfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:36.858968 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c6dd1-9b0f-448e-aade-d1318e520e2c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:36.859079 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae969b47-1bbd-45a1-870a-c08b8cf6ccfb None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:36.859240 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae969b47-1bbd-45a1-870a-c08b8cf6ccfb None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:36.859332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c6dd1-9b0f-448e-aade-d1318e520e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:36.859518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae969b47-1bbd-45a1-870a-c08b8cf6ccfb None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:36.859610 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c6dd1-9b0f-448e-aade-d1318e520e2c None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:36.860239 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c6dd1-9b0f-448e-aade-d1318e520e2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:36.860331 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae969b47-1bbd-45a1-870a-c08b8cf6ccfb None admin] RBAC: Adding query filter params (name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:36.860550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c6dd1-9b0f-448e-aade-d1318e520e2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:36.860641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae969b47-1bbd-45a1-870a-c08b8cf6ccfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:36.861932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae969b47-1bbd-45a1-870a-c08b8cf6ccfb None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:36.862031 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c6dd1-9b0f-448e-aade-d1318e520e2c None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:36.878381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 239/498] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:12:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:36.878381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae969b47-1bbd-45a1-870a-c08b8cf6ccfb None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:36.878631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 260/499] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:36.878631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12401e1-44c7-471f-9ac7-508dabc9cc5d None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:36.878631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12401e1-44c7-471f-9ac7-508dabc9cc5d None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:36.878631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12401e1-44c7-471f-9ac7-508dabc9cc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:36.894314 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-e12401e1-44c7-471f-9ac7-508dabc9cc5d None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:36.895546 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e12401e1-44c7-471f-9ac7-508dabc9cc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:36.970394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 261/500] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:36.970394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12401e1-44c7-471f-9ac7-508dabc9cc5d None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:36.994338 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:37.001565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae969b47-1bbd-45a1-870a-c08b8cf6ccfb None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:37.018758 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:37.037333 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479aad92-19d4-4540-a507-285570acb5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:37.037981 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479aad92-19d4-4540-a507-285570acb5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:37.038286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479aad92-19d4-4540-a507-285570acb5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:37.038572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479aad92-19d4-4540-a507-285570acb5e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:37.064708 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12401e1-44c7-471f-9ac7-508dabc9cc5d None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:37.099588 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a023c81-a331-4930-a79a-8f605d8835bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:37.100395 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a023c81-a331-4930-a79a-8f605d8835bd None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:37.100671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a023c81-a331-4930-a79a-8f605d8835bd None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:37.100942 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a023c81-a331-4930-a79a-8f605d8835bd None admin] PATH_INFO: `/v3/users/glance` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:37.108447 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a023c81-a331-4930-a79a-8f605d8835bd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:37.108957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479aad92-19d4-4540-a507-285570acb5e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:37.109900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a023c81-a331-4930-a79a-8f605d8835bd None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:37.110396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479aad92-19d4-4540-a507-285570acb5e2 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:37.124853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-0a023c81-a331-4930-a79a-8f605d8835bd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 15:12:37.124853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:37.124853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:37.124853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:37.124853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:37.124853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:37.124853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:37.124853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:37.124853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:37.124853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:37.124853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:37.124853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:37.124853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:12:37.124853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:12:37.124853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:12:37.124853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:37.125769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:37.126715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:37.126715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:37.126715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:12:37.126715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:12:37.126715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:12:37.126715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:12:37.126715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:12:37.126715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:12:37.126715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 15:12:37.126715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:37.126715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 262/501] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:37.127503 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a023c81-a331-4930-a79a-8f605d8835bd None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:37.204687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:37.212140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 240/502] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:37.217689 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479aad92-19d4-4540-a507-285570acb5e2 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:37.228937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a023c81-a331-4930-a79a-8f605d8835bd None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:37.241315 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:37.261432 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af4651-ca3e-4cbd-9200-cf3bad9aad08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:37.262327 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af4651-ca3e-4cbd-9200-cf3bad9aad08 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:37.262614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af4651-ca3e-4cbd-9200-cf3bad9aad08 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:37.262889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af4651-ca3e-4cbd-9200-cf3bad9aad08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:37.274067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af4651-ca3e-4cbd-9200-cf3bad9aad08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:37.275563 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af4651-ca3e-4cbd-9200-cf3bad9aad08 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:37.292754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-82af4651-ca3e-4cbd-9200-cf3bad9aad08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:37.292754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:37.292754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:37.292754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:37.292754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:37.292754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:37.292754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:37.292754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:37.292754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:37.292754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:37.292754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:37.292754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:37.292754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:37.292754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:37.292754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:37.294935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:37.294935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:37.294935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:37.294935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:37.294935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:37.294935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:37.294935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:37.294935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:37.294935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:37.294935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:37.294935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:37.294935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:37.294935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:37.294935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:37.294935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:37.294935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:37.294935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:37.296052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:37.296052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:37.296052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:37.296052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:37.296052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 263/503] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:37.296052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc66de9-1993-4cf5-a999-8364e5bceb64 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:37.296052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc66de9-1993-4cf5-a999-8364e5bceb64 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:37.296052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc66de9-1993-4cf5-a999-8364e5bceb64 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:37.296930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 264/504] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:37.297618 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc66de9-1993-4cf5-a999-8364e5bceb64 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:37.313212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479aad92-19d4-4540-a507-285570acb5e2 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:37.348733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6605a53-e736-4484-9fb7-78b85f6b6c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:37.349584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6605a53-e736-4484-9fb7-78b85f6b6c4e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:37.349867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6605a53-e736-4484-9fb7-78b85f6b6c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:37.350167 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6605a53-e736-4484-9fb7-78b85f6b6c4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:37.360728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6605a53-e736-4484-9fb7-78b85f6b6c4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:37.368648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-b6605a53-e736-4484-9fb7-78b85f6b6c4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:37.368648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:37.368648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:37.368648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:37.368648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:37.368648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:37.368648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:37.368648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:37.368648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:37.368648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:37.368648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:37.368648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:37.368648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:37.368648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:37.368648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:37.368648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:37.369615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:37.369615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:37.369615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:37.369615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:37.369615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:37.369615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:37.369615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:37.369615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:37.369615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:37.369615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:37.369615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:37.369615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:37.369615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:37.369615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:37.369615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:37.369615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:37.370446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:37.370446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:37.370446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:37.370446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:37.370446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:37.370446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:37.370446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:37.370446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:37.370446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:37.370446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:37.370446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:37.370446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:37.370446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 241/505] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:37.371520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6605a53-e736-4484-9fb7-78b85f6b6c4e None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:37.401024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc66de9-1993-4cf5-a999-8364e5bceb64 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:37.425145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:37.434502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d10585-f5f7-4efb-b791-dcf22c95eafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:37.435175 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d10585-f5f7-4efb-b791-dcf22c95eafe None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:37.435460 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d10585-f5f7-4efb-b791-dcf22c95eafe None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:37.437280 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d10585-f5f7-4efb-b791-dcf22c95eafe None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:37.437280 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d10585-f5f7-4efb-b791-dcf22c95eafe None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:37.437280 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d10585-f5f7-4efb-b791-dcf22c95eafe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:37.438334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d10585-f5f7-4efb-b791-dcf22c95eafe None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:37.454980 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 265/506] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Jul 20 15:12:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:37.455772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d10585-f5f7-4efb-b791-dcf22c95eafe None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:37.469398 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6605a53-e736-4484-9fb7-78b85f6b6c4e None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:37.479988 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:37.511331 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9fa05e-81b1-4510-9fa2-2fae53cc6c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:37.512226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fa05e-81b1-4510-9fa2-2fae53cc6c86 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:37.512597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fa05e-81b1-4510-9fa2-2fae53cc6c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:37.512875 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fa05e-81b1-4510-9fa2-2fae53cc6c86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:37.523399 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9fa05e-81b1-4510-9fa2-2fae53cc6c86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:37.524852 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9fa05e-81b1-4510-9fa2-2fae53cc6c86 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:37.542223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-2b9fa05e-81b1-4510-9fa2-2fae53cc6c86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:37.542223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:37.542223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:37.542223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:37.542223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:37.542223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:37.542223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:37.542223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:37.542223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:37.542223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:37.542223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:37.542223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:37.542223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:37.542223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:37.542223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:37.543280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:37.543280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:37.543280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:37.543280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:37.543280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:37.543280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:37.543280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:37.543280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:37.543280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:37.543280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:37.543280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:37.543280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:37.543280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:37.543280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:37.543280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:37.543280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:37.543280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:37.544572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:37.544572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:37.544572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:37.544572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:37.544572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 242/507] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:37.544906 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48193dc-7cc4-49fc-b74c-bde7c429f1c3 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:37.545180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48193dc-7cc4-49fc-b74c-bde7c429f1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:37.545461 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48193dc-7cc4-49fc-b74c-bde7c429f1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:37.557398 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d10585-f5f7-4efb-b791-dcf22c95eafe None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:37.562689 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-a48193dc-7cc4-49fc-b74c-bde7c429f1c3 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:37.563879 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a48193dc-7cc4-49fc-b74c-bde7c429f1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:37.590345 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d937e0-d5c8-44ad-95ad-183f0ab6dc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:37.591005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d937e0-d5c8-44ad-95ad-183f0ab6dc39 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:37.591283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d937e0-d5c8-44ad-95ad-183f0ab6dc39 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:37.591559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d937e0-d5c8-44ad-95ad-183f0ab6dc39 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:37.592269 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d937e0-d5c8-44ad-95ad-183f0ab6dc39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:37.592577 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d937e0-d5c8-44ad-95ad-183f0ab6dc39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:37.593943 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d937e0-d5c8-44ad-95ad-183f0ab6dc39 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:37.605799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 266/508] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:37.606448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d937e0-d5c8-44ad-95ad-183f0ab6dc39 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:37.633698 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 243/509] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:37.634573 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48193dc-7cc4-49fc-b74c-bde7c429f1c3 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:37.668518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:37.688130 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:37.705671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d937e0-d5c8-44ad-95ad-183f0ab6dc39 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:37.732497 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48193dc-7cc4-49fc-b74c-bde7c429f1c3 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:37.740287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5dea5-fa1b-4956-82e0-ed1043a940f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:37.740959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5dea5-fa1b-4956-82e0-ed1043a940f1 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:37.741245 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5dea5-fa1b-4956-82e0-ed1043a940f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:37.741524 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5dea5-fa1b-4956-82e0-ed1043a940f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:37.742280 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5dea5-fa1b-4956-82e0-ed1043a940f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:37.742602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5dea5-fa1b-4956-82e0-ed1043a940f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:37.743974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5dea5-fa1b-4956-82e0-ed1043a940f1 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:37.751822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 267/510] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:12:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:37.752606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5dea5-fa1b-4956-82e0-ed1043a940f1 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:37.770558 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265008bc-f7a0-422e-bc11-4fdb8b8e62a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:37.771204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265008bc-f7a0-422e-bc11-4fdb8b8e62a2 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:37.771487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265008bc-f7a0-422e-bc11-4fdb8b8e62a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:37.771760 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265008bc-f7a0-422e-bc11-4fdb8b8e62a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:37.838080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265008bc-f7a0-422e-bc11-4fdb8b8e62a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:37.839598 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265008bc-f7a0-422e-bc11-4fdb8b8e62a2 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:37.851242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5dea5-fa1b-4956-82e0-ed1043a940f1 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:37.874183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:37.884746 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874e95d-617a-472c-8cf9-b6c563fba88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:37.885457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874e95d-617a-472c-8cf9-b6c563fba88a None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:37.885738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874e95d-617a-472c-8cf9-b6c563fba88a None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:37.886012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874e95d-617a-472c-8cf9-b6c563fba88a None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:37.886760 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874e95d-617a-472c-8cf9-b6c563fba88a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:37.887069 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874e95d-617a-472c-8cf9-b6c563fba88a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:37.888446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874e95d-617a-472c-8cf9-b6c563fba88a None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:37.899864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 268/511] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:37.900645 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874e95d-617a-472c-8cf9-b6c563fba88a None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:37.905714 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:37.937853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 244/512] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:37.938794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265008bc-f7a0-422e-bc11-4fdb8b8e62a2 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:38.000342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874e95d-617a-472c-8cf9-b6c563fba88a None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:38.033589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78dea2-37a3-4da3-8ad7-c13d364e0ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:38.034495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78dea2-37a3-4da3-8ad7-c13d364e0ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:38.034779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78dea2-37a3-4da3-8ad7-c13d364e0ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:38.035054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78dea2-37a3-4da3-8ad7-c13d364e0ea7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:38.035549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265008bc-f7a0-422e-bc11-4fdb8b8e62a2 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:38.042939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78dea2-37a3-4da3-8ad7-c13d364e0ea7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:38.044394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78dea2-37a3-4da3-8ad7-c13d364e0ea7 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:38.058838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-7a78dea2-37a3-4da3-8ad7-c13d364e0ea7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 15:12:38.058838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:38.058838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:38.058838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:38.058838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:38.058838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:38.058838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:38.058838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:38.058838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:38.058838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:38.058838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:38.058838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:38.058838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:12:38.058838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:12:38.058838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:12:38.058838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:38.059939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:38.061068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:38.061068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:38.061068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:12:38.061068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:12:38.061068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:12:38.061068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:12:38.061068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:12:38.061068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:12:38.061068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 15:12:38.061068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:38.063518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 269/513] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:38.063518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78dea2-37a3-4da3-8ad7-c13d364e0ea7 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:38.069176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39121d-40e5-4a01-a7ea-caaeb0affd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:38.069970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39121d-40e5-4a01-a7ea-caaeb0affd1d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:38.070282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39121d-40e5-4a01-a7ea-caaeb0affd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:38.070561 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39121d-40e5-4a01-a7ea-caaeb0affd1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:38.073300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:38.081740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c39121d-40e5-4a01-a7ea-caaeb0affd1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:38.083236 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c39121d-40e5-4a01-a7ea-caaeb0affd1d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:38.101720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-2c39121d-40e5-4a01-a7ea-caaeb0affd1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:38.101720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:38.101720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:38.101720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:38.101720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:38.101720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:38.101720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:38.101720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:38.101720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:38.101720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:38.101720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:38.101720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:38.101720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:38.101720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:38.101720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:38.102734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:38.102734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:38.102734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:38.102734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:38.102734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:38.102734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:38.102734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:38.102734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:38.102734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:38.102734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:38.102734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:38.102734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:38.102734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:38.102734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:38.102734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:38.102734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:38.102734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:38.103645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:38.103645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:38.103645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:38.103645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:38.103645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 245/514] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:38.104363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39121d-40e5-4a01-a7ea-caaeb0affd1d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:38.130867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:38.168672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78dea2-37a3-4da3-8ad7-c13d364e0ea7 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:38.201523 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf14f43-6b6a-4866-8cec-4d515970d4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:38.202349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf14f43-6b6a-4866-8cec-4d515970d4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:38.202639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf14f43-6b6a-4866-8cec-4d515970d4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:38.202920 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf14f43-6b6a-4866-8cec-4d515970d4e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:38.203179 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39121d-40e5-4a01-a7ea-caaeb0affd1d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:38.213226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf14f43-6b6a-4866-8cec-4d515970d4e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:38.214723 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf14f43-6b6a-4866-8cec-4d515970d4e4 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:38.231876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-edf14f43-6b6a-4866-8cec-4d515970d4e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:38.231876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:38.231876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:38.231876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:38.231876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:38.231876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:38.231876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:38.231876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:38.231876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:38.231876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:38.231876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:38.231876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:38.231876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:38.231876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:38.231876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:38.232738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:38.232738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:38.232738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:38.232738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:38.232738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:38.232738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:38.232738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:38.232738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:38.232738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:38.232738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:38.232738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:38.232738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:38.232738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:38.232738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:38.232738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:38.232738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:38.232738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:38.233596 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:38.233596 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:38.233596 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:38.233596 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:38.233596 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 270/515] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:38.234144 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf14f43-6b6a-4866-8cec-4d515970d4e4 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:38.241113 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09c4ef-429e-4f47-8dd4-2bf5842382cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:38.241906 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09c4ef-429e-4f47-8dd4-2bf5842382cd None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:38.242192 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09c4ef-429e-4f47-8dd4-2bf5842382cd None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:38.242501 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09c4ef-429e-4f47-8dd4-2bf5842382cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:38.249499 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09c4ef-429e-4f47-8dd4-2bf5842382cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:38.251048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09c4ef-429e-4f47-8dd4-2bf5842382cd None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:38.267065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-ea09c4ef-429e-4f47-8dd4-2bf5842382cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:38.267065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:38.267065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:38.267065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:38.267065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:38.267065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:38.267065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:38.267065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:38.267065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:38.267065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:38.267065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:38.267065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:38.267065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:12:38.267065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:12:38.267065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:12:38.268133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:12:38.268133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:38.268133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:38.268133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:38.268133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:38.268133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:38.268133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:38.268133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:38.268133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:38.268133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:38.268133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:38.268133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:38.268133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:38.268133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:38.268133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:38.268133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:38.268133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:38.269312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:12:38.269312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:12:38.269312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:12:38.269312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:12:38.269312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:12:38.269312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:12:38.269312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:38.269312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:38.269312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 246/516] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:38.269312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09c4ef-429e-4f47-8dd4-2bf5842382cd None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:38.316818 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:38.340885 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf14f43-6b6a-4866-8cec-4d515970d4e4 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:38.357086 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:38.367533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09c4ef-429e-4f47-8dd4-2bf5842382cd None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:38.374911 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453bcb5c-7c77-4a8a-96c8-7a69d12c00ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:38.375786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bcb5c-7c77-4a8a-96c8-7a69d12c00ec None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:38.381699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bcb5c-7c77-4a8a-96c8-7a69d12c00ec None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:38.381699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bcb5c-7c77-4a8a-96c8-7a69d12c00ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:38.386941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453bcb5c-7c77-4a8a-96c8-7a69d12c00ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:38.388407 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453bcb5c-7c77-4a8a-96c8-7a69d12c00ec None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:38.401392 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a941649-1a63-423b-a846-226506d6462c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:38.402054 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a941649-1a63-423b-a846-226506d6462c None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:38.402365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a941649-1a63-423b-a846-226506d6462c None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:38.402647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a941649-1a63-423b-a846-226506d6462c None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:38.403483 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a941649-1a63-423b-a846-226506d6462c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:38.403798 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a941649-1a63-423b-a846-226506d6462c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:38.405231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a941649-1a63-423b-a846-226506d6462c None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:38.406032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-453bcb5c-7c77-4a8a-96c8-7a69d12c00ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:38.406032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:38.406032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:38.406032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:38.406032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:38.406032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:38.406032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:38.406032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:38.406032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:38.406032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:38.406032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:38.406032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:38.406032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:38.406032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:38.406032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:38.407085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:38.407085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:38.407085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:38.407085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:38.407085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:38.407085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:38.407085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:38.407085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:38.407085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:38.407085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:38.407085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:38.407085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:38.407085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:38.407085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:38.407085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:38.407085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:38.407085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:38.408023 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:38.408023 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:38.408023 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:38.408023 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:38.408023 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 271/517] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:38.408609 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453bcb5c-7c77-4a8a-96c8-7a69d12c00ec None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:38.449211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 247/518] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Jul 20 15:12:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:38.450060 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a941649-1a63-423b-a846-226506d6462c None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:38.512228 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453bcb5c-7c77-4a8a-96c8-7a69d12c00ec None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:38.545935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cc3b0-a398-4d46-bc2e-5cf8067b0a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:38.546634 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cc3b0-a398-4d46-bc2e-5cf8067b0a74 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:38.546916 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cc3b0-a398-4d46-bc2e-5cf8067b0a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:38.547189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cc3b0-a398-4d46-bc2e-5cf8067b0a74 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:38.547900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223cc3b0-a398-4d46-bc2e-5cf8067b0a74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:38.548202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223cc3b0-a398-4d46-bc2e-5cf8067b0a74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:38.549589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223cc3b0-a398-4d46-bc2e-5cf8067b0a74 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:38.556166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a941649-1a63-423b-a846-226506d6462c None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:38.561256 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 272/519] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:38.562034 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cc3b0-a398-4d46-bc2e-5cf8067b0a74 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:38.563983 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:38.589624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:38.591394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c914a7-a6b4-4d70-a6fd-758cc59bbd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:38.592042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c914a7-a6b4-4d70-a6fd-758cc59bbd1b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:38.592325 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c914a7-a6b4-4d70-a6fd-758cc59bbd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:38.592612 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c914a7-a6b4-4d70-a6fd-758cc59bbd1b None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:38.593336 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c914a7-a6b4-4d70-a6fd-758cc59bbd1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:38.593646 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c914a7-a6b4-4d70-a6fd-758cc59bbd1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:38.595070 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c914a7-a6b4-4d70-a6fd-758cc59bbd1b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:38.606576 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 248/520] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:38.607484 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c914a7-a6b4-4d70-a6fd-758cc59bbd1b None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:38.661332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cc3b0-a398-4d46-bc2e-5cf8067b0a74 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:38.694353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e771952-69b8-40b9-9ef1-c80fcb99c12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:38.695008 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e771952-69b8-40b9-9ef1-c80fcb99c12b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:38.695290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e771952-69b8-40b9-9ef1-c80fcb99c12b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:38.695667 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e771952-69b8-40b9-9ef1-c80fcb99c12b None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:38.696450 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e771952-69b8-40b9-9ef1-c80fcb99c12b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:38.696760 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e771952-69b8-40b9-9ef1-c80fcb99c12b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:38.698200 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e771952-69b8-40b9-9ef1-c80fcb99c12b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:38.704528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c914a7-a6b4-4d70-a6fd-758cc59bbd1b None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:38.716605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 273/521] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:12:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:38.717408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e771952-69b8-40b9-9ef1-c80fcb99c12b None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:38.740939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887615ac-1a6c-4b6c-abc2-09ce4f194e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:38.741596 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887615ac-1a6c-4b6c-abc2-09ce4f194e2d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:38.741878 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887615ac-1a6c-4b6c-abc2-09ce4f194e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:38.742210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887615ac-1a6c-4b6c-abc2-09ce4f194e2d None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:38.742959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887615ac-1a6c-4b6c-abc2-09ce4f194e2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:38.743265 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887615ac-1a6c-4b6c-abc2-09ce4f194e2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:38.744690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887615ac-1a6c-4b6c-abc2-09ce4f194e2d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:38.757032 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 249/522] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:38.757725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887615ac-1a6c-4b6c-abc2-09ce4f194e2d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:38.783738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:38.808799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:38.820130 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e771952-69b8-40b9-9ef1-c80fcb99c12b None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:38.853791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3fda2-4bd7-41d2-bb79-47807932529d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:38.854482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3fda2-4bd7-41d2-bb79-47807932529d None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:38.854774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3fda2-4bd7-41d2-bb79-47807932529d None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:38.855059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3fda2-4bd7-41d2-bb79-47807932529d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:38.856132 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887615ac-1a6c-4b6c-abc2-09ce4f194e2d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:38.891678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d001fd-17d2-4ed9-afc0-ba7e0e7d74b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:38.892338 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d001fd-17d2-4ed9-afc0-ba7e0e7d74b8 None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:38.892620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d001fd-17d2-4ed9-afc0-ba7e0e7d74b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:38.892892 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d001fd-17d2-4ed9-afc0-ba7e0e7d74b8 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:38.893588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d001fd-17d2-4ed9-afc0-ba7e0e7d74b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:38.895749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d001fd-17d2-4ed9-afc0-ba7e0e7d74b8 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:38.921237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3fda2-4bd7-41d2-bb79-47807932529d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:38.922718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3fda2-4bd7-41d2-bb79-47807932529d None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:38.973569 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 250/523] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:12:38 2022] POST /identity/v3/users => generated 247 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:12:38.974496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d001fd-17d2-4ed9-afc0-ba7e0e7d74b8 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:39.027737 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 274/524] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:39.028639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3fda2-4bd7-41d2-bb79-47807932529d None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:39.039785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:39.069132 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:39.088373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d001fd-17d2-4ed9-afc0-ba7e0e7d74b8 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:39.127927 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7963d6d-06f7-4e5f-ac30-a2d1cda2e0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:39.129600 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7963d6d-06f7-4e5f-ac30-a2d1cda2e0ec None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:39.129854 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7963d6d-06f7-4e5f-ac30-a2d1cda2e0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:39.130122 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7963d6d-06f7-4e5f-ac30-a2d1cda2e0ec None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:39.138006 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7963d6d-06f7-4e5f-ac30-a2d1cda2e0ec None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:39.139597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7963d6d-06f7-4e5f-ac30-a2d1cda2e0ec None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:39.155089 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3fda2-4bd7-41d2-bb79-47807932529d None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:39.156397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-b7963d6d-06f7-4e5f-ac30-a2d1cda2e0ec None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 15:12:39.156397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:39.156397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:39.156397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:39.156397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:39.156397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:39.156397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:39.156397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:39.156397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:39.156397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:39.156397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:39.156397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:39.156397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:12:39.156397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:12:39.156397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:12:39.156397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:39.157608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:39.158662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:39.158662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:39.158662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:12:39.158662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:12:39.158662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:12:39.158662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:12:39.158662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:12:39.158662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:12:39.158662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 15:12:39.158662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:39.158662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 251/525] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:39.158662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7963d6d-06f7-4e5f-ac30-a2d1cda2e0ec None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:39.201986 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10445585-f8ee-4709-b15c-d6d967f485bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:39.202747 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10445585-f8ee-4709-b15c-d6d967f485bc None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:39.203034 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10445585-f8ee-4709-b15c-d6d967f485bc None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:39.203312 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10445585-f8ee-4709-b15c-d6d967f485bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:39.204144 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10445585-f8ee-4709-b15c-d6d967f485bc None admin] RBAC: Adding query filter params (role.id=d820b1d3fa4144aca575c04645c099bc, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=a4a24d5a375945779fa626fae25ac237) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:39.204461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10445585-f8ee-4709-b15c-d6d967f485bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:39.205924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10445585-f8ee-4709-b15c-d6d967f485bc None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:39.214347 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 275/526] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:12:39 2022] GET /identity/v3/role_assignments?user.id=a4a24d5a375945779fa626fae25ac237&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=d820b1d3fa4144aca575c04645c099bc => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:39.214347 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10445585-f8ee-4709-b15c-d6d967f485bc None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:39.279584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7963d6d-06f7-4e5f-ac30-a2d1cda2e0ec None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:39.293024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:39.321216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab39c3-c185-4088-b247-c052e9bd9a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:39.322043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab39c3-c185-4088-b247-c052e9bd9a07 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:39.322355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab39c3-c185-4088-b247-c052e9bd9a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:39.322635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab39c3-c185-4088-b247-c052e9bd9a07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:39.333313 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab39c3-c185-4088-b247-c052e9bd9a07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:39.336110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab39c3-c185-4088-b247-c052e9bd9a07 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:39.354969 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10445585-f8ee-4709-b15c-d6d967f485bc None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:39.356176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-43ab39c3-c185-4088-b247-c052e9bd9a07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:39.356176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:39.356176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:39.356176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:39.356176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:39.356176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:39.356176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:39.356176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:39.356176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:39.356176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:39.356176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:39.356176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:39.356176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:12:39.356176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:12:39.356176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:12:39.357028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:12:39.357028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:39.357028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:39.357028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:39.357028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:39.357028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:39.357028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:39.357028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:39.357028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:39.357028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:39.357028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:39.357028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:39.357028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:39.357028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:39.357028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:39.357028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:39.357028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:39.357894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:12:39.357894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:12:39.357894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:12:39.357894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:12:39.357894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:12:39.357894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:12:39.357894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:39.357894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:39.358994 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 252/527] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:39.362303 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab39c3-c185-4088-b247-c052e9bd9a07 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:39.370654 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:39.400678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee7534-9ac6-4205-83ae-c300df5148af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:39.401493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee7534-9ac6-4205-83ae-c300df5148af None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:39.401963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee7534-9ac6-4205-83ae-c300df5148af None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:39.402252 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee7534-9ac6-4205-83ae-c300df5148af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:39.416111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee7534-9ac6-4205-83ae-c300df5148af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:39.418244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee7534-9ac6-4205-83ae-c300df5148af None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:39.440517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-56ee7534-9ac6-4205-83ae-c300df5148af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:39.440517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:39.440517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:39.440517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:39.440517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:39.440517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:39.440517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:39.440517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:39.440517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:39.440517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:39.440517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:39.440517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:39.440517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:39.440517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:39.440517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:39.441455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:39.441455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:39.441455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:39.441455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:39.441455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:39.441455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:39.441455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:39.441455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:39.441455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:39.441455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:39.441455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:39.441455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:39.441455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:39.441455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:39.441455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:39.441455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:39.441455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:39.442884 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:39.442884 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:39.442884 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:39.442884 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:39.442884 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 276/528] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:39.446305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee7534-9ac6-4205-83ae-c300df5148af None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:39.484518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab39c3-c185-4088-b247-c052e9bd9a07 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:39.524313 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1c2bf-bb02-4529-a682-277fa267de43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:39.525726 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1c2bf-bb02-4529-a682-277fa267de43 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:39.526227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1c2bf-bb02-4529-a682-277fa267de43 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:39.526730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1c2bf-bb02-4529-a682-277fa267de43 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:39.528123 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1c2bf-bb02-4529-a682-277fa267de43 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:39.528624 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1c2bf-bb02-4529-a682-277fa267de43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:39.530901 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1c2bf-bb02-4529-a682-277fa267de43 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:39.558541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee7534-9ac6-4205-83ae-c300df5148af None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:39.559781 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 253/529] 10.222.0.78 () {56 vars in 1139 bytes} [Wed Jul 20 15:12:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:39.560767 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:39.562252 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1c2bf-bb02-4529-a682-277fa267de43 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:39.602935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c959d-214a-45c3-914b-42b948b7a86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:39.603744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c959d-214a-45c3-914b-42b948b7a86a None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:39.604095 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c959d-214a-45c3-914b-42b948b7a86a None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:39.604440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c959d-214a-45c3-914b-42b948b7a86a None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:39.605540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c959d-214a-45c3-914b-42b948b7a86a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:39.605917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c959d-214a-45c3-914b-42b948b7a86a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:39.608555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c959d-214a-45c3-914b-42b948b7a86a None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:39.624520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 277/530] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:12:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:39.625582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c959d-214a-45c3-914b-42b948b7a86a None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:39.636630 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:39.692302 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1c2bf-bb02-4529-a682-277fa267de43 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:39.730634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874776b-a63c-47bf-bdd6-b3c61d54974d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:39.731631 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874776b-a63c-47bf-bdd6-b3c61d54974d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:39.732085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874776b-a63c-47bf-bdd6-b3c61d54974d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:39.732531 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874776b-a63c-47bf-bdd6-b3c61d54974d None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:39.733428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1874776b-a63c-47bf-bdd6-b3c61d54974d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:39.733907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1874776b-a63c-47bf-bdd6-b3c61d54974d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:39.735491 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1874776b-a63c-47bf-bdd6-b3c61d54974d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:39.749374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 254/531] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:39.750288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874776b-a63c-47bf-bdd6-b3c61d54974d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:39.768885 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c959d-214a-45c3-914b-42b948b7a86a None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:39.815408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7d658-bca1-46b2-93e7-e8a0797c6325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:39.816708 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7d658-bca1-46b2-93e7-e8a0797c6325 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:39.817439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7d658-bca1-46b2-93e7-e8a0797c6325 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:39.818388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7d658-bca1-46b2-93e7-e8a0797c6325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:39.858197 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:39.862339 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874776b-a63c-47bf-bdd6-b3c61d54974d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:39.889507 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:39.899068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac7d658-bca1-46b2-93e7-e8a0797c6325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:39.901012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac7d658-bca1-46b2-93e7-e8a0797c6325 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:39.906376 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91091b67-3bbd-4b10-9c39-3bd3e9c4c3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:39.907041 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91091b67-3bbd-4b10-9c39-3bd3e9c4c3df None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:39.907324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91091b67-3bbd-4b10-9c39-3bd3e9c4c3df None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:39.907601 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91091b67-3bbd-4b10-9c39-3bd3e9c4c3df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:39.908428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91091b67-3bbd-4b10-9c39-3bd3e9c4c3df None admin] RBAC: Adding query filter params (role.id=d820b1d3fa4144aca575c04645c099bc, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=b6c13596c2df438294bdac7c52ae8cb1) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:39.908737 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91091b67-3bbd-4b10-9c39-3bd3e9c4c3df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:39.910177 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91091b67-3bbd-4b10-9c39-3bd3e9c4c3df None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:39.918495 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 255/532] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:12:39 2022] GET /identity/v3/role_assignments?user.id=b6c13596c2df438294bdac7c52ae8cb1&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=d820b1d3fa4144aca575c04645c099bc => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:39.919297 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91091b67-3bbd-4b10-9c39-3bd3e9c4c3df None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:40.017111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 278/533] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:40.023205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7d658-bca1-46b2-93e7-e8a0797c6325 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:40.058396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91091b67-3bbd-4b10-9c39-3bd3e9c4c3df None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:40.122181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e7972-7bc4-4bb4-9f7a-d4c705d1b01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:40.123020 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e7972-7bc4-4bb4-9f7a-d4c705d1b01a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:40.123305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e7972-7bc4-4bb4-9f7a-d4c705d1b01a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:40.123584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e7972-7bc4-4bb4-9f7a-d4c705d1b01a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:40.137826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e7972-7bc4-4bb4-9f7a-d4c705d1b01a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:40.143399 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:40.145489 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e7972-7bc4-4bb4-9f7a-d4c705d1b01a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:40.164170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-0f4e7972-7bc4-4bb4-9f7a-d4c705d1b01a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:40.164170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:40.164170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:40.164170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:40.164170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:40.164170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:40.164170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:40.164170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:40.164170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:40.164170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:40.164170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:40.164170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:40.164170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:12:40.164170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:12:40.164170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:12:40.165055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:12:40.165055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:40.165055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:40.165055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:40.165055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:40.165055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:40.165055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:40.165055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:40.165055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:40.165055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:40.165055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:40.165055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:40.165055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:40.165055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:40.165055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:40.165055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:40.165055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:40.165904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:12:40.165904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:12:40.165904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:12:40.165904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:12:40.165904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:12:40.165904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:12:40.165904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:40.165904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:40.166347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 256/534] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:40.172160 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e7972-7bc4-4bb4-9f7a-d4c705d1b01a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:40.179446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7d658-bca1-46b2-93e7-e8a0797c6325 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:40.224990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ef08d-43ce-4514-acb1-fb6f7de4c3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:40.225850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ef08d-43ce-4514-acb1-fb6f7de4c3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:40.226138 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ef08d-43ce-4514-acb1-fb6f7de4c3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:40.226445 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ef08d-43ce-4514-acb1-fb6f7de4c3b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:40.253953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ef08d-43ce-4514-acb1-fb6f7de4c3b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:40.255991 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ef08d-43ce-4514-acb1-fb6f7de4c3b0 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:40.262694 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:40.287401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-312ef08d-43ce-4514-acb1-fb6f7de4c3b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:40.287401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:40.287401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:40.287401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:40.287401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:40.287401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:40.287401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:40.287401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:40.287401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:40.287401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:40.287401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:40.287401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:40.287401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:40.287401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:40.287401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:40.288306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:40.288306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:40.288306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:40.288306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:40.288306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:40.288306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:40.288306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:40.288306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:40.288306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:40.288306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:40.288306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:40.288306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:40.288306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:40.288306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:40.288306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:40.288306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:40.288306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:40.289237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:40.289237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:40.289237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:40.289237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:40.289237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 279/535] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:40.296400 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ef08d-43ce-4514-acb1-fb6f7de4c3b0 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:40.369284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e7972-7bc4-4bb4-9f7a-d4c705d1b01a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:40.454992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b3874-84ec-416c-a96e-d0e714716fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:40.455686 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b3874-84ec-416c-a96e-d0e714716fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:40.458305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b3874-84ec-416c-a96e-d0e714716fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:40.458305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b3874-84ec-416c-a96e-d0e714716fa4 None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:40.458620 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ef08d-43ce-4514-acb1-fb6f7de4c3b0 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:40.458767 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b3874-84ec-416c-a96e-d0e714716fa4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:40.458767 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b3874-84ec-416c-a96e-d0e714716fa4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:40.459552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b3874-84ec-416c-a96e-d0e714716fa4 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:40.474863 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:40.493364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958c360-980f-4a46-baac-e1ec7553caa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:40.494032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958c360-980f-4a46-baac-e1ec7553caa7 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:40.494334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958c360-980f-4a46-baac-e1ec7553caa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:40.498290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958c360-980f-4a46-baac-e1ec7553caa7 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:40.498290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958c360-980f-4a46-baac-e1ec7553caa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:40.498290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958c360-980f-4a46-baac-e1ec7553caa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:40.498290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958c360-980f-4a46-baac-e1ec7553caa7 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:40.501935 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 257/536] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:12:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:40.508629 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b3874-84ec-416c-a96e-d0e714716fa4 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:40.509973 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 280/537] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:40.515022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958c360-980f-4a46-baac-e1ec7553caa7 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:40.667533 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958c360-980f-4a46-baac-e1ec7553caa7 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:40.708179 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b3874-84ec-416c-a96e-d0e714716fa4 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:40.719769 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:40.735703 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d2881-98fb-452d-88a8-ae8f07632d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:40.736512 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d2881-98fb-452d-88a8-ae8f07632d38 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:40.737652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d2881-98fb-452d-88a8-ae8f07632d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:40.737652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d2881-98fb-452d-88a8-ae8f07632d38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:40.749331 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d2881-98fb-452d-88a8-ae8f07632d38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:40.750825 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d2881-98fb-452d-88a8-ae8f07632d38 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:40.772775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-ea0d2881-98fb-452d-88a8-ae8f07632d38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:40.772775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:40.772775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:40.772775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:40.772775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:40.772775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:40.772775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:40.772775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:40.772775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:40.772775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:40.772775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:40.772775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:40.772775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:12:40.772775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:12:40.772775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:12:40.773683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:12:40.773683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:40.773683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:40.773683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:40.773683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:40.773683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:40.773683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:40.773683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:40.773683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:40.773683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:40.773683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:40.773683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:40.773683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:40.773683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:40.773683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:40.773683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:40.773683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:40.774602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:12:40.774602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:12:40.774602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:12:40.774602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:12:40.774602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:12:40.774602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:12:40.774602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:40.774602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:40.778529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 281/538] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:40.787220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c51a6-f4c8-4ce5-9d37-9283bca23d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:40.787885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c51a6-f4c8-4ce5-9d37-9283bca23d38 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:40.788170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c51a6-f4c8-4ce5-9d37-9283bca23d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:40.788446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c51a6-f4c8-4ce5-9d37-9283bca23d38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:40.789279 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c51a6-f4c8-4ce5-9d37-9283bca23d38 None admin] RBAC: Adding query filter params (role.id=d820b1d3fa4144aca575c04645c099bc, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=b51e9bdab4164d87b7b0b11c505c2c0a) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:40.789590 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c51a6-f4c8-4ce5-9d37-9283bca23d38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:40.791076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c51a6-f4c8-4ce5-9d37-9283bca23d38 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:40.793668 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d2881-98fb-452d-88a8-ae8f07632d38 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:40.808485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 258/539] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:12:40 2022] GET /identity/v3/role_assignments?user.id=b51e9bdab4164d87b7b0b11c505c2c0a&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=d820b1d3fa4144aca575c04645c099bc => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:12:40.838288 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:40.910285 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d2881-98fb-452d-88a8-ae8f07632d38 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:40.953984 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd507bb5-ab84-4d66-a475-46fcbc42c694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:40.954618 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd507bb5-ab84-4d66-a475-46fcbc42c694 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:40.954902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd507bb5-ab84-4d66-a475-46fcbc42c694 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:40.955184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd507bb5-ab84-4d66-a475-46fcbc42c694 None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:40.955990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd507bb5-ab84-4d66-a475-46fcbc42c694 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:40.956302 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd507bb5-ab84-4d66-a475-46fcbc42c694 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:40.957831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd507bb5-ab84-4d66-a475-46fcbc42c694 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:40.977261 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 282/540] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:12:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:40.983353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c51a6-f4c8-4ce5-9d37-9283bca23d38 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:41.100687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c51a6-f4c8-4ce5-9d37-9283bca23d38 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:41.141740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc427177-ddfb-44bf-890f-81e735be69e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:41.142432 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc427177-ddfb-44bf-890f-81e735be69e1 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:41.142721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc427177-ddfb-44bf-890f-81e735be69e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:41.142999 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc427177-ddfb-44bf-890f-81e735be69e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:41.143837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc427177-ddfb-44bf-890f-81e735be69e1 None admin] RBAC: Adding query filter params (role.id=d820b1d3fa4144aca575c04645c099bc, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=b57530f18d9c4d328f8014db736d8de9) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:41.144147 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc427177-ddfb-44bf-890f-81e735be69e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:41.145614 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc427177-ddfb-44bf-890f-81e735be69e1 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:41.151583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 259/541] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:12:40 2022] GET /identity/v3/role_assignments?user.id=b57530f18d9c4d328f8014db736d8de9&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=d820b1d3fa4144aca575c04645c099bc => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:41.424852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14af8fc-9f67-4522-9bf4-1f9d279bdf39 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:41.425358 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14af8fc-9f67-4522-9bf4-1f9d279bdf39 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:41.425432 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14af8fc-9f67-4522-9bf4-1f9d279bdf39 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:41.427713 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 283/542] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:41.432906 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f393703-d8c9-43ba-933b-9c436eacdb57 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:41.433312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f393703-d8c9-43ba-933b-9c436eacdb57 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:41.433496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f393703-d8c9-43ba-933b-9c436eacdb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:41.452290 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-9f393703-d8c9-43ba-933b-9c436eacdb57 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:41.453828 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-9f393703-d8c9-43ba-933b-9c436eacdb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:41.485118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6ebe2-cb9b-4099-9246-624f50a454b8 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:41.485436 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6ebe2-cb9b-4099-9246-624f50a454b8 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:41.485684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6ebe2-cb9b-4099-9246-624f50a454b8 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:41.487264 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 284/543] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:12:41.490011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:41.492702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19161ea6-1606-4eda-b2ab-772cb9c32d41 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:41.492996 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19161ea6-1606-4eda-b2ab-772cb9c32d41 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:41.493275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19161ea6-1606-4eda-b2ab-772cb9c32d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:41.522043 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-19161ea6-1606-4eda-b2ab-772cb9c32d41 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:41.523402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-19161ea6-1606-4eda-b2ab-772cb9c32d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:41.566390 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 260/544] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:41.602563 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f393703-d8c9-43ba-933b-9c436eacdb57 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:41.654622 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 285/545] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 15:12:41.706584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19161ea6-1606-4eda-b2ab-772cb9c32d41 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:41.762643 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:41.773575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f393703-d8c9-43ba-933b-9c436eacdb57 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:41.825047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665a3db-feed-4b75-b4bf-842d78c75166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:41.825847 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665a3db-feed-4b75-b4bf-842d78c75166 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:41.826129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665a3db-feed-4b75-b4bf-842d78c75166 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:41.826427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665a3db-feed-4b75-b4bf-842d78c75166 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:41.855631 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665a3db-feed-4b75-b4bf-842d78c75166 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:41.865817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-6665a3db-feed-4b75-b4bf-842d78c75166 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:41.865817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:41.865817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:41.865817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:41.865817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:41.865817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:41.865817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:41.865817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:41.865817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:41.865817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:41.865817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:41.865817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:41.865817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:41.865817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:41.865817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:41.865817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:41.866730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:41.866730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:41.866730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:41.866730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:41.866730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:41.866730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:41.866730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:41.866730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:41.866730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:41.866730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:41.866730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:41.866730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:41.866730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:41.866730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:41.866730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:41.866730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:41.867574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:41.867574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:41.867574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:41.867574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:41.867574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:41.867574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:41.867574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:41.867574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:41.867574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:41.867574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:41.867574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:41.867574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:41.867574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 261/546] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:41.877647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665a3db-feed-4b75-b4bf-842d78c75166 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:41.906825 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19161ea6-1606-4eda-b2ab-772cb9c32d41 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:41.926872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:41.966671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb725aa-dd11-4f66-8190-cf91a2c930d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:41.967323 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb725aa-dd11-4f66-8190-cf91a2c930d6 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:41.967604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb725aa-dd11-4f66-8190-cf91a2c930d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:41.967881 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb725aa-dd11-4f66-8190-cf91a2c930d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:41.989484 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb725aa-dd11-4f66-8190-cf91a2c930d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:41.997532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-bfb725aa-dd11-4f66-8190-cf91a2c930d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:41.997532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:41.997532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:41.997532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:41.997532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:41.997532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:41.997532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:41.997532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:41.997532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:41.997532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:41.997532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:41.997532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:41.997532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:41.997532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:41.997532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:41.997532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:41.998466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:41.998466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:41.998466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:41.998466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:41.998466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:41.998466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:41.998466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:41.998466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:41.998466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:41.998466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:41.998466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:41.998466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:41.998466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:41.998466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:41.998466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:41.998466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:41.999352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:41.999352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:41.999352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:41.999352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:41.999352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:41.999352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:41.999352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:41.999352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:41.999352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:41.999352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:41.999352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:41.999352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:42.005778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 286/547] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:42.010536 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb725aa-dd11-4f66-8190-cf91a2c930d6 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:42.072853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665a3db-feed-4b75-b4bf-842d78c75166 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:42.144142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cee26-cea0-42ea-919f-44525faad717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:42.145356 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cee26-cea0-42ea-919f-44525faad717 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:42.145637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cee26-cea0-42ea-919f-44525faad717 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:42.145910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cee26-cea0-42ea-919f-44525faad717 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:42.146696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cee26-cea0-42ea-919f-44525faad717 None admin] RBAC: Adding query filter params (name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:42.147003 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cee26-cea0-42ea-919f-44525faad717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:42.148366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cee26-cea0-42ea-919f-44525faad717 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:42.149856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:42.165427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 262/548] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:12:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:42.175207 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cee26-cea0-42ea-919f-44525faad717 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:42.177245 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb725aa-dd11-4f66-8190-cf91a2c930d6 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:42.212088 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293f527-80ad-4c1e-8ae6-114364cd11bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:42.212808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293f527-80ad-4c1e-8ae6-114364cd11bd None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:42.213092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293f527-80ad-4c1e-8ae6-114364cd11bd None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:42.213362 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293f527-80ad-4c1e-8ae6-114364cd11bd None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:42.214099 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293f527-80ad-4c1e-8ae6-114364cd11bd None admin] RBAC: Adding query filter params (name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:42.214438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293f527-80ad-4c1e-8ae6-114364cd11bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:42.215969 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293f527-80ad-4c1e-8ae6-114364cd11bd None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:42.240926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 287/549] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:12:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:42.251200 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293f527-80ad-4c1e-8ae6-114364cd11bd None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:42.307378 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:42.330420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cee26-cea0-42ea-919f-44525faad717 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:42.395510 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6d383-2ec4-4efd-91ea-2ce917a758cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:42.396325 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6d383-2ec4-4efd-91ea-2ce917a758cc None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:42.396613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6d383-2ec4-4efd-91ea-2ce917a758cc None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:42.396905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6d383-2ec4-4efd-91ea-2ce917a758cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:42.409779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293f527-80ad-4c1e-8ae6-114364cd11bd None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:42.416659 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6d383-2ec4-4efd-91ea-2ce917a758cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:42.418388 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6d383-2ec4-4efd-91ea-2ce917a758cc None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:42.439672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-4cb6d383-2ec4-4efd-91ea-2ce917a758cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:42.439672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:42.439672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:42.439672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:42.439672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:42.439672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:42.439672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:42.439672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:42.439672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:42.439672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:42.439672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:42.439672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:42.439672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:42.439672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:42.439672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:42.441024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:42.441024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:42.441024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:42.441024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:42.441024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:42.441024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:42.441024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:42.441024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:42.441024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:42.441024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:42.441024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:42.441024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:42.441024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:42.441024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:42.441024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:42.441024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:42.441024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:42.441942 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:42.441942 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:42.441942 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:42.441942 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:42.441942 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 263/550] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:42.442680 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc50951-deea-4a3b-9f12-8593c441ab01 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:42.442776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc50951-deea-4a3b-9f12-8593c441ab01 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:42.443000 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc50951-deea-4a3b-9f12-8593c441ab01 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:42.445890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 264/551] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:42.447242 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc50951-deea-4a3b-9f12-8593c441ab01 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:42.451169 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fca42-3116-43ae-b021-f73ef51f901b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:42.452507 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fca42-3116-43ae-b021-f73ef51f901b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:42.452993 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fca42-3116-43ae-b021-f73ef51f901b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:42.453485 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fca42-3116-43ae-b021-f73ef51f901b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:42.465270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5fca42-3116-43ae-b021-f73ef51f901b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:42.467075 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5fca42-3116-43ae-b021-f73ef51f901b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:42.475972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:42.485515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-3c5fca42-3116-43ae-b021-f73ef51f901b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:42.485515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:42.485515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:42.485515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:42.485515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:42.485515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:42.485515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:42.485515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:42.485515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:42.485515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:42.485515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:42.485515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:42.485515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:42.485515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:42.485515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:42.504693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:42.504693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:42.504693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:42.504693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:42.504693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:42.504693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:42.504693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:42.504693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:42.504693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:42.504693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:42.504693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:42.504693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:42.504693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:42.504693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:42.504693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:42.504693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:42.504693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:42.505589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:42.505589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:42.505589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:42.505589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:42.505589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 288/552] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:42.505589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcffc85-41c5-4c51-ae03-19cb3af25951 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:42.505589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcffc85-41c5-4c51-ae03-19cb3af25951 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:42.505589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcffc85-41c5-4c51-ae03-19cb3af25951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:42.512258 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-4fcffc85-41c5-4c51-ae03-19cb3af25951 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:42.513759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-4fcffc85-41c5-4c51-ae03-19cb3af25951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:42.567691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc50951-deea-4a3b-9f12-8593c441ab01 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:42.602509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 289/553] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:42.603409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcffc85-41c5-4c51-ae03-19cb3af25951 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:42.610746 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573d8fe-6eec-4274-80b2-23b9c81e0849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:42.611737 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573d8fe-6eec-4274-80b2-23b9c81e0849 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:42.612233 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573d8fe-6eec-4274-80b2-23b9c81e0849 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:42.612705 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573d8fe-6eec-4274-80b2-23b9c81e0849 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:42.614778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573d8fe-6eec-4274-80b2-23b9c81e0849 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:42.615336 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573d8fe-6eec-4274-80b2-23b9c81e0849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:42.617122 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573d8fe-6eec-4274-80b2-23b9c81e0849 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:42.620376 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:42.633618 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 265/554] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:42.635569 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573d8fe-6eec-4274-80b2-23b9c81e0849 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:42.724909 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcffc85-41c5-4c51-ae03-19cb3af25951 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:42.754620 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:42.760331 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463123d-e32f-4dd2-a8cb-be724363569e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:42.761149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463123d-e32f-4dd2-a8cb-be724363569e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:42.761332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463123d-e32f-4dd2-a8cb-be724363569e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:42.761591 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463123d-e32f-4dd2-a8cb-be724363569e None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:42.762459 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463123d-e32f-4dd2-a8cb-be724363569e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:42.767588 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463123d-e32f-4dd2-a8cb-be724363569e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:42.767588 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463123d-e32f-4dd2-a8cb-be724363569e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:42.771455 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573d8fe-6eec-4274-80b2-23b9c81e0849 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:42.780403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 290/555] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:42.781942 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463123d-e32f-4dd2-a8cb-be724363569e None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:42.811042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2b21b-db2b-4727-b535-605ccbcfb312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:42.812273 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2b21b-db2b-4727-b535-605ccbcfb312 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:42.812814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2b21b-db2b-4727-b535-605ccbcfb312 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:42.813292 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2b21b-db2b-4727-b535-605ccbcfb312 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:42.824769 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2b21b-db2b-4727-b535-605ccbcfb312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:42.833578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-fde2b21b-db2b-4727-b535-605ccbcfb312 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:42.833578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:42.833578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:42.833578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:42.833578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:42.833578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:42.833578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:42.833578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:42.833578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:42.833578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:42.833578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:42.833578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:42.833578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:42.833578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:42.833578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:42.833578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:42.834533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:42.834533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:42.834533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:42.834533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:42.834533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:42.834533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:42.834533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:42.834533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:42.834533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:42.834533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:42.834533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:42.834533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:42.834533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:42.834533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:42.834533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:42.834533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:42.835400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:42.835400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:42.835400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:42.835400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:42.835400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:42.835400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:42.835400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:42.835400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:42.835400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:42.835400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:42.835400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:42.835400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:42.838307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 266/556] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:42.839374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2b21b-db2b-4727-b535-605ccbcfb312 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:42.894662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463123d-e32f-4dd2-a8cb-be724363569e None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:42.925228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:42.929961 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf60ede-4653-4a62-b250-6eb8a0cb28ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:42.930924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf60ede-4653-4a62-b250-6eb8a0cb28ed None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:42.933369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf60ede-4653-4a62-b250-6eb8a0cb28ed None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:42.933369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf60ede-4653-4a62-b250-6eb8a0cb28ed None admin] PATH_INFO: `/v3/users/swift` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:42.940269 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf60ede-4653-4a62-b250-6eb8a0cb28ed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:42.942238 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf60ede-4653-4a62-b250-6eb8a0cb28ed None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:42.956798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-dbf60ede-4653-4a62-b250-6eb8a0cb28ed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:12:42.956798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:42.956798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:42.956798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:42.956798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:42.956798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:42.956798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:42.956798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:42.956798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:42.956798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:42.956798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:42.956798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:42.956798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:12:42.956798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:12:42.956798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:12:42.956798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:42.958396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:42.959808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:42.959808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:42.959808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:12:42.959808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:12:42.959808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:12:42.959808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:12:42.959808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:12:42.959808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:12:42.959808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:12:42.959808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:42.959808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 291/557] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:12:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:42.959808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf60ede-4653-4a62-b250-6eb8a0cb28ed None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:42.992122 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2b21b-db2b-4727-b535-605ccbcfb312 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:43.024794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:43.032336 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301830cf-ff71-4170-a6ac-d0ddeedba565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:43.033290 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301830cf-ff71-4170-a6ac-d0ddeedba565 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:43.033704 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301830cf-ff71-4170-a6ac-d0ddeedba565 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:43.034038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301830cf-ff71-4170-a6ac-d0ddeedba565 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:43.042391 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301830cf-ff71-4170-a6ac-d0ddeedba565 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:43.043944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301830cf-ff71-4170-a6ac-d0ddeedba565 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:43.059611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-301830cf-ff71-4170-a6ac-d0ddeedba565 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:12:43.059611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:43.059611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:43.059611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:43.059611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:43.059611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:43.059611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:43.059611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:43.059611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:43.059611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:43.059611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:43.059611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:43.059611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:12:43.059611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:12:43.059611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:12:43.059611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:43.061066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:43.062442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:43.062442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:43.062442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:12:43.062442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:12:43.062442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:12:43.062442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:12:43.062442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:12:43.062442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:12:43.062442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:12:43.062442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:43.068834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 267/558] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Jul 20 15:12:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:43.069801 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301830cf-ff71-4170-a6ac-d0ddeedba565 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:43.101313 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf60ede-4653-4a62-b250-6eb8a0cb28ed None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:43.137271 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a1a1e-5a85-490f-b87a-248b08b4f443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:43.137975 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a1a1e-5a85-490f-b87a-248b08b4f443 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:43.138271 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a1a1e-5a85-490f-b87a-248b08b4f443 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:43.138586 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a1a1e-5a85-490f-b87a-248b08b4f443 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:43.139334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a1a1e-5a85-490f-b87a-248b08b4f443 None admin] RBAC: Adding query filter params (name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:43.139642 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a1a1e-5a85-490f-b87a-248b08b4f443 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:43.141014 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a1a1e-5a85-490f-b87a-248b08b4f443 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:43.149131 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 292/559] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:12:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:43.149935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a1a1e-5a85-490f-b87a-248b08b4f443 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:43.180131 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301830cf-ff71-4170-a6ac-d0ddeedba565 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:43.197692 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:43.215839 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3038064-01f5-43e8-ada5-024858f27bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:43.216571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3038064-01f5-43e8-ada5-024858f27bce None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:43.216856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3038064-01f5-43e8-ada5-024858f27bce None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:43.217134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3038064-01f5-43e8-ada5-024858f27bce None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:43.218039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3038064-01f5-43e8-ada5-024858f27bce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:43.218381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3038064-01f5-43e8-ada5-024858f27bce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:43.219898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3038064-01f5-43e8-ada5-024858f27bce None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:43.239014 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 268/560] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Jul 20 15:12:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:43.244437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6737aed-f90a-4473-9b66-9c435e47607f None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:43.244755 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6737aed-f90a-4473-9b66-9c435e47607f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:43.245065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6737aed-f90a-4473-9b66-9c435e47607f None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:43.247058 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 269/561] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:43.251297 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6737aed-f90a-4473-9b66-9c435e47607f None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:43.264087 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a1a1e-5a85-490f-b87a-248b08b4f443 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:43.278317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:43.299054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884790ac-809f-4eb9-b447-988d062d4d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:43.299710 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884790ac-809f-4eb9-b447-988d062d4d5e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:43.299994 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884790ac-809f-4eb9-b447-988d062d4d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:43.300276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884790ac-809f-4eb9-b447-988d062d4d5e None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:43.301652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884790ac-809f-4eb9-b447-988d062d4d5e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:43.301961 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884790ac-809f-4eb9-b447-988d062d4d5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:43.303423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884790ac-809f-4eb9-b447-988d062d4d5e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:43.321287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 293/562] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Jul 20 15:12:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:43.321926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884790ac-809f-4eb9-b447-988d062d4d5e None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:43.364616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6737aed-f90a-4473-9b66-9c435e47607f None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:43.400622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4074e3-112d-44b4-8b90-c6d2307763ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:43.401853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4074e3-112d-44b4-8b90-c6d2307763ec None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:43.402138 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4074e3-112d-44b4-8b90-c6d2307763ec None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:43.402447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4074e3-112d-44b4-8b90-c6d2307763ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:43.414247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4074e3-112d-44b4-8b90-c6d2307763ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:43.415929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4074e3-112d-44b4-8b90-c6d2307763ec None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:43.429137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884790ac-809f-4eb9-b447-988d062d4d5e None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:43.437526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-4d4074e3-112d-44b4-8b90-c6d2307763ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:43.437526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:43.437526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:43.437526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:43.437526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:43.437526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:43.437526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:43.437526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:43.437526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:43.437526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:43.437526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:43.437526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:43.437526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:43.437526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:43.437526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:43.438667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:43.438667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:43.438667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:43.438667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:43.438667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:43.438667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:43.438667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:43.438667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:43.438667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:43.438667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:43.438667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:43.438667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:43.438667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:43.438667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:43.438667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:43.438667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:43.438667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:43.439733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:43.439733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:43.439733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:43.439733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:43.439733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 270/563] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:43.441350 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2bf68-c253-4eae-b372-bfd3c1e09806 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:43.441639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2bf68-c253-4eae-b372-bfd3c1e09806 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:43.441916 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2bf68-c253-4eae-b372-bfd3c1e09806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:43.460841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-e5f2bf68-c253-4eae-b372-bfd3c1e09806 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:43.462060 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e5f2bf68-c253-4eae-b372-bfd3c1e09806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:43.473015 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432d1d8-21da-44a1-9c58-df66dcdc47e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:43.473803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432d1d8-21da-44a1-9c58-df66dcdc47e1 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:43.474096 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432d1d8-21da-44a1-9c58-df66dcdc47e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:43.474431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432d1d8-21da-44a1-9c58-df66dcdc47e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:43.484810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:43.514669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:43.531563 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 271/564] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:43.532325 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2bf68-c253-4eae-b372-bfd3c1e09806 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:43.544068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432d1d8-21da-44a1-9c58-df66dcdc47e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:43.545565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432d1d8-21da-44a1-9c58-df66dcdc47e1 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:43.636815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2bf68-c253-4eae-b372-bfd3c1e09806 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:43.643426 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 294/565] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:43.644254 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432d1d8-21da-44a1-9c58-df66dcdc47e1 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:43.672851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56080b1e-19a1-4a1e-ac4b-d29ec5b8bfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:43.673638 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56080b1e-19a1-4a1e-ac4b-d29ec5b8bfe1 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:43.673917 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56080b1e-19a1-4a1e-ac4b-d29ec5b8bfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:43.674185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56080b1e-19a1-4a1e-ac4b-d29ec5b8bfe1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:43.684703 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56080b1e-19a1-4a1e-ac4b-d29ec5b8bfe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:43.686126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56080b1e-19a1-4a1e-ac4b-d29ec5b8bfe1 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:43.703394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-56080b1e-19a1-4a1e-ac4b-d29ec5b8bfe1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:43.703394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:43.703394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:43.703394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:43.703394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:43.703394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:43.703394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:43.703394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:43.703394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:43.703394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:43.703394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:43.703394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:43.703394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:43.703394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:43.703394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:43.704420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:43.704420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:43.704420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:43.704420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:43.704420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:43.704420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:43.704420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:43.704420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:43.704420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:43.704420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:43.704420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:43.704420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:43.704420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:43.704420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:43.704420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:43.704420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:43.704420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:43.705328 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:43.705328 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:43.705328 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:43.705328 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:43.705328 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 272/566] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:43.705780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ebc8f-caa8-43b5-9f98-6cc8f9bc9ffc None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:43.706120 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ebc8f-caa8-43b5-9f98-6cc8f9bc9ffc None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:43.706361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ebc8f-caa8-43b5-9f98-6cc8f9bc9ffc None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:43.707652 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 273/567] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:43.708458 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ebc8f-caa8-43b5-9f98-6cc8f9bc9ffc None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:43.717255 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:43.719811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:43.748500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432d1d8-21da-44a1-9c58-df66dcdc47e1 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:43.784696 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef863a7-83f0-45ba-beae-43ca6c6bcfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:43.785825 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef863a7-83f0-45ba-beae-43ca6c6bcfb3 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:43.786114 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef863a7-83f0-45ba-beae-43ca6c6bcfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:43.786423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef863a7-83f0-45ba-beae-43ca6c6bcfb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:43.787152 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef863a7-83f0-45ba-beae-43ca6c6bcfb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:43.787460 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef863a7-83f0-45ba-beae-43ca6c6bcfb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:43.788852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef863a7-83f0-45ba-beae-43ca6c6bcfb3 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:43.800842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 295/568] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:43.801638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef863a7-83f0-45ba-beae-43ca6c6bcfb3 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:43.820369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ebc8f-caa8-43b5-9f98-6cc8f9bc9ffc None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:43.855157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68941da7-6903-448b-b288-4a0505595b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:43.855951 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68941da7-6903-448b-b288-4a0505595b67 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:43.856304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68941da7-6903-448b-b288-4a0505595b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:43.856579 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68941da7-6903-448b-b288-4a0505595b67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:43.866703 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68941da7-6903-448b-b288-4a0505595b67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:43.874211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-68941da7-6903-448b-b288-4a0505595b67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:43.874211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:43.874211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:43.874211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:43.874211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:43.874211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:43.874211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:43.874211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:43.874211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:43.874211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:43.874211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:43.874211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:43.874211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:43.874211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:43.874211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:43.874211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:43.875241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:43.875241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:43.875241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:43.875241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:43.875241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:43.875241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:43.875241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:43.875241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:43.875241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:43.875241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:43.875241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:43.875241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:43.875241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:43.875241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:43.875241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:43.875241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:43.876106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:43.876106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:43.876106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:43.876106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:43.876106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:43.876106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:43.876106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:43.876106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:43.876106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:43.876106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:43.876106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:43.876106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:43.876106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 274/569] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:43.876828 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68941da7-6903-448b-b288-4a0505595b67 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:43.903093 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef863a7-83f0-45ba-beae-43ca6c6bcfb3 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:43.930942 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:43.936548 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77641b7c-146b-4e85-a0e7-e6cd46200986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:43.938782 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77641b7c-146b-4e85-a0e7-e6cd46200986 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:43.938782 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77641b7c-146b-4e85-a0e7-e6cd46200986 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:43.938782 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77641b7c-146b-4e85-a0e7-e6cd46200986 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:43.943704 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:43.947753 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77641b7c-146b-4e85-a0e7-e6cd46200986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:43.949198 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77641b7c-146b-4e85-a0e7-e6cd46200986 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:43.967157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-77641b7c-146b-4e85-a0e7-e6cd46200986 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:43.967157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:43.967157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:43.967157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:43.967157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:43.967157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:43.967157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:43.967157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:43.967157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:43.967157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:43.967157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:43.967157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:43.967157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:43.967157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:43.967157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:43.968248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:43.968248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:43.968248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:43.968248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:43.968248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:43.968248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:43.968248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:43.968248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:43.968248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:43.968248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:43.968248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:43.968248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:43.968248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:43.968248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:43.968248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:43.968248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:43.968248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:43.969208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:43.969208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:43.969208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:43.969208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:43.969208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 296/570] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:43.969832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a012973-15e2-4b3f-a37c-874c4596e313 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:43.970791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a012973-15e2-4b3f-a37c-874c4596e313 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:43.970944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a012973-15e2-4b3f-a37c-874c4596e313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:43.979011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68941da7-6903-448b-b288-4a0505595b67 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:43.987179 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-9a012973-15e2-4b3f-a37c-874c4596e313 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:43.988879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9a012973-15e2-4b3f-a37c-874c4596e313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:44.014088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02bcb81-74d9-49a5-ac54-fb3574f29f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:44.014818 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bcb81-74d9-49a5-ac54-fb3574f29f82 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:44.015096 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bcb81-74d9-49a5-ac54-fb3574f29f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:44.015378 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bcb81-74d9-49a5-ac54-fb3574f29f82 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:44.016091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02bcb81-74d9-49a5-ac54-fb3574f29f82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:44.016395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02bcb81-74d9-49a5-ac54-fb3574f29f82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:44.017754 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02bcb81-74d9-49a5-ac54-fb3574f29f82 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:44.029839 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 275/571] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:44.030649 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02bcb81-74d9-49a5-ac54-fb3574f29f82 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:44.061190 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 297/572] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:44.061950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a012973-15e2-4b3f-a37c-874c4596e313 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:44.139553 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02bcb81-74d9-49a5-ac54-fb3574f29f82 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:44.161935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a012973-15e2-4b3f-a37c-874c4596e313 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:44.166744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:44.175511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809dc51f-c47d-46fe-bb9b-ec74296a84d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:44.176830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dc51f-c47d-46fe-bb9b-ec74296a84d2 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:44.184756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:44.184907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dc51f-c47d-46fe-bb9b-ec74296a84d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:44.184907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dc51f-c47d-46fe-bb9b-ec74296a84d2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:44.184907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809dc51f-c47d-46fe-bb9b-ec74296a84d2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:44.186003 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809dc51f-c47d-46fe-bb9b-ec74296a84d2 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:44.198612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64122c-b40c-4ecc-a09d-5ada206868ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:44.199276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64122c-b40c-4ecc-a09d-5ada206868ed None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:44.199559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64122c-b40c-4ecc-a09d-5ada206868ed None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:44.199837 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64122c-b40c-4ecc-a09d-5ada206868ed None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:44.200283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-809dc51f-c47d-46fe-bb9b-ec74296a84d2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 15:12:44.200283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:44.200283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:44.200283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:44.200283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:44.200283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:44.200283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:44.200283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:44.200283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:44.200283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:44.200283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:44.200283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:44.200283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:12:44.200283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:12:44.200283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:12:44.200283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:44.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:44.202757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:44.202757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:44.202757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:12:44.202757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:12:44.202757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:12:44.202757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:12:44.202757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:12:44.202757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:12:44.202757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 15:12:44.202757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:44.203504 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64122c-b40c-4ecc-a09d-5ada206868ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:44.203504 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64122c-b40c-4ecc-a09d-5ada206868ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:44.203658 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 276/573] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:44.203748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64122c-b40c-4ecc-a09d-5ada206868ed None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:44.203832 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809dc51f-c47d-46fe-bb9b-ec74296a84d2 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:44.215180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 298/574] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:12:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:44.215180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64122c-b40c-4ecc-a09d-5ada206868ed None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:44.305354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809dc51f-c47d-46fe-bb9b-ec74296a84d2 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:44.311519 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64122c-b40c-4ecc-a09d-5ada206868ed None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:44.338149 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca926e2-3363-4652-b4ef-477891d73c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:44.338902 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca926e2-3363-4652-b4ef-477891d73c17 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:44.339184 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca926e2-3363-4652-b4ef-477891d73c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:44.339456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca926e2-3363-4652-b4ef-477891d73c17 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:44.340208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca926e2-3363-4652-b4ef-477891d73c17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:44.340549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca926e2-3363-4652-b4ef-477891d73c17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:44.341904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca926e2-3363-4652-b4ef-477891d73c17 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:44.353229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 277/575] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:44.354004 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca926e2-3363-4652-b4ef-477891d73c17 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:44.381644 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfaae6d-f364-4a5b-b8d9-e83c0a264bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:44.382470 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfaae6d-f364-4a5b-b8d9-e83c0a264bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:44.382758 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfaae6d-f364-4a5b-b8d9-e83c0a264bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:44.383039 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfaae6d-f364-4a5b-b8d9-e83c0a264bd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:44.389766 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfaae6d-f364-4a5b-b8d9-e83c0a264bd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:44.391265 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfaae6d-f364-4a5b-b8d9-e83c0a264bd2 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:44.404438 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:44.405992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-fcfaae6d-f364-4a5b-b8d9-e83c0a264bd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:44.405992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:44.405992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:44.405992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:44.405992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:44.405992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:44.405992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:44.405992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:44.405992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:44.405992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:44.405992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:44.405992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:44.405992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:12:44.405992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:12:44.405992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:12:44.406928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:12:44.406928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:44.406928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:44.406928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:44.406928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:44.406928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:44.406928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:44.406928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:44.406928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:44.406928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:44.406928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:44.406928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:44.406928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:44.406928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:44.406928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:44.406928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:44.406928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:44.407848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:12:44.407848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:12:44.407848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:12:44.407848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:12:44.407848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:12:44.407848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:12:44.407848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:44.407848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:44.407848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 299/576] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:44.408720 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfaae6d-f364-4a5b-b8d9-e83c0a264bd2 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:44.441228 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:44.455515 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca926e2-3363-4652-b4ef-477891d73c17 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:44.493372 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c739162-d857-4743-a310-76235b9c6b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:44.494161 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c739162-d857-4743-a310-76235b9c6b98 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:44.494466 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c739162-d857-4743-a310-76235b9c6b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:44.494741 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c739162-d857-4743-a310-76235b9c6b98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:44.505487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c739162-d857-4743-a310-76235b9c6b98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:44.508106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfaae6d-f364-4a5b-b8d9-e83c0a264bd2 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:44.513205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-3c739162-d857-4743-a310-76235b9c6b98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:44.513205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:44.513205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:44.513205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:44.513205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:44.513205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:44.513205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:44.513205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:44.513205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:44.513205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:44.513205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:44.513205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:44.513205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:44.513205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:44.513205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:44.513205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:44.514357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:44.514357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:44.514357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:44.514357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:44.514357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:44.514357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:44.514357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:44.514357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:44.514357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:44.514357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:44.514357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:44.514357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:44.514357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:44.514357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:44.514357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:44.514357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:44.515249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:44.515249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:44.515249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:44.515249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:44.515249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:44.515249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:44.515249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:44.515249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:44.515249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:44.515249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:44.515249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:44.515249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:44.515249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 278/577] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:44.515249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c739162-d857-4743-a310-76235b9c6b98 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:44.546761 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25037bd2-3103-4a77-8cec-cb3994f8b2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:44.547431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25037bd2-3103-4a77-8cec-cb3994f8b2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:44.547709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25037bd2-3103-4a77-8cec-cb3994f8b2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:44.547985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25037bd2-3103-4a77-8cec-cb3994f8b2e0 None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:44.548810 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25037bd2-3103-4a77-8cec-cb3994f8b2e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:44.549119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25037bd2-3103-4a77-8cec-cb3994f8b2e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:44.550629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25037bd2-3103-4a77-8cec-cb3994f8b2e0 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:44.567168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 300/578] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Jul 20 15:12:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:44.567900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25037bd2-3103-4a77-8cec-cb3994f8b2e0 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:44.612550 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c739162-d857-4743-a310-76235b9c6b98 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:44.621063 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:44.645416 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbb1cd-d618-4584-91f9-aba8bb88010a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:44.646203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbb1cd-d618-4584-91f9-aba8bb88010a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:44.646514 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbb1cd-d618-4584-91f9-aba8bb88010a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:44.646790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbb1cd-d618-4584-91f9-aba8bb88010a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:44.656594 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:44.658064 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cbb1cd-d618-4584-91f9-aba8bb88010a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:44.659554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cbb1cd-d618-4584-91f9-aba8bb88010a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:44.666324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25037bd2-3103-4a77-8cec-cb3994f8b2e0 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:44.677546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-28cbb1cd-d618-4584-91f9-aba8bb88010a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:44.677546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:44.677546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:44.677546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:44.677546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:44.677546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:44.677546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:44.677546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:44.677546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:44.677546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:44.677546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:44.677546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:44.677546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:44.677546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:44.677546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:44.678500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:44.678500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:44.678500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:44.678500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:44.678500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:44.678500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:44.678500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:44.678500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:44.678500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:44.678500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:44.678500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:44.678500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:44.678500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:44.678500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:44.678500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:44.678500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:44.678500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:44.679616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:44.679616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:44.679616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:44.679616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:44.679616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 279/579] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:44.679616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbb1cd-d618-4584-91f9-aba8bb88010a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:44.701835 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e08c63-f978-436c-8064-9a4f1e798957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:44.702653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e08c63-f978-436c-8064-9a4f1e798957 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:44.702937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e08c63-f978-436c-8064-9a4f1e798957 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:44.703215 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e08c63-f978-436c-8064-9a4f1e798957 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:44.710320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e08c63-f978-436c-8064-9a4f1e798957 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:44.711793 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e08c63-f978-436c-8064-9a4f1e798957 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:44.726318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-30e08c63-f978-436c-8064-9a4f1e798957 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:44.726318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:44.726318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:44.726318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:44.726318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:44.726318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:44.726318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:44.726318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:44.726318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:44.726318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:44.726318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:44.726318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:44.726318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:12:44.726318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:12:44.726318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:12:44.727334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:12:44.727334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:44.727334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:44.727334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:44.727334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:44.727334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:44.727334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:44.727334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:44.727334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:44.727334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:44.727334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:44.727334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:44.727334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:44.727334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:44.727334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:44.727334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:44.727334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:44.728263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:12:44.728263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:12:44.728263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:12:44.728263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:12:44.728263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:12:44.728263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:12:44.728263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:44.728263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:44.728263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 301/580] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:44.728263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e08c63-f978-436c-8064-9a4f1e798957 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:44.784179 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbb1cd-d618-4584-91f9-aba8bb88010a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:44.816975 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf6e23-d9eb-405e-85d6-b93220240e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:44.817684 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf6e23-d9eb-405e-85d6-b93220240e78 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:44.817909 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf6e23-d9eb-405e-85d6-b93220240e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:44.818181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf6e23-d9eb-405e-85d6-b93220240e78 None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:44.818986 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bf6e23-d9eb-405e-85d6-b93220240e78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:44.819292 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bf6e23-d9eb-405e-85d6-b93220240e78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:44.820716 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bf6e23-d9eb-405e-85d6-b93220240e78 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:44.825963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e08c63-f978-436c-8064-9a4f1e798957 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:44.832932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 280/581] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:12:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:44.833521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf6e23-d9eb-405e-85d6-b93220240e78 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:44.854505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:44.861396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66641c-fe46-4725-82fc-aa38c657f98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:44.862073 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66641c-fe46-4725-82fc-aa38c657f98f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:44.866064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66641c-fe46-4725-82fc-aa38c657f98f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:44.866064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66641c-fe46-4725-82fc-aa38c657f98f None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:44.866064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66641c-fe46-4725-82fc-aa38c657f98f None admin] RBAC: Adding query filter params (name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:44.866064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66641c-fe46-4725-82fc-aa38c657f98f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:44.866064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66641c-fe46-4725-82fc-aa38c657f98f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:44.873095 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 302/582] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:12:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:44.873872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66641c-fe46-4725-82fc-aa38c657f98f None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:44.889799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:44.932632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf6e23-d9eb-405e-85d6-b93220240e78 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:44.967555 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75905140-b7c4-4275-9e87-c79d06e4337d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:44.968341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75905140-b7c4-4275-9e87-c79d06e4337d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:44.968622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75905140-b7c4-4275-9e87-c79d06e4337d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:44.968898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75905140-b7c4-4275-9e87-c79d06e4337d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:44.972698 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66641c-fe46-4725-82fc-aa38c657f98f None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:44.978828 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75905140-b7c4-4275-9e87-c79d06e4337d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:44.980251 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75905140-b7c4-4275-9e87-c79d06e4337d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:44.997613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-75905140-b7c4-4275-9e87-c79d06e4337d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:44.997613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:44.997613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:44.997613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:44.997613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:44.997613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:44.997613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:44.997613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:44.997613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:44.997613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:44.997613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:44.997613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:44.997613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:44.997613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:44.997613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:44.998551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:44.998551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:44.998551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:44.998551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:44.998551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:44.998551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:44.998551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:44.998551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:44.998551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:44.998551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:44.998551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:44.998551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:44.998551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:44.998551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:44.998551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:44.998551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:44.998551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:44.999508 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:44.999508 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:44.999508 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:44.999508 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:44.999508 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 281/583] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:44.999789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75905140-b7c4-4275-9e87-c79d06e4337d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:45.008123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd05f8-f937-4097-86c8-a4bf910429f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:45.008783 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd05f8-f937-4097-86c8-a4bf910429f1 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:45.009062 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd05f8-f937-4097-86c8-a4bf910429f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:45.009336 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd05f8-f937-4097-86c8-a4bf910429f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:45.010048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd05f8-f937-4097-86c8-a4bf910429f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:45.010384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd05f8-f937-4097-86c8-a4bf910429f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:45.011768 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd05f8-f937-4097-86c8-a4bf910429f1 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:45.023450 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 303/584] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:45.024279 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd05f8-f937-4097-86c8-a4bf910429f1 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:45.075206 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:45.098082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:45.107570 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75905140-b7c4-4275-9e87-c79d06e4337d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:45.121112 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd05f8-f937-4097-86c8-a4bf910429f1 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:45.141438 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad19f2e-2f05-401d-b553-2c508f4f74d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:45.142094 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad19f2e-2f05-401d-b553-2c508f4f74d3 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:45.142409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad19f2e-2f05-401d-b553-2c508f4f74d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:45.142689 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad19f2e-2f05-401d-b553-2c508f4f74d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:45.143472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad19f2e-2f05-401d-b553-2c508f4f74d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:45.143786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad19f2e-2f05-401d-b553-2c508f4f74d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:45.145748 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad19f2e-2f05-401d-b553-2c508f4f74d3 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:45.155652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1c383-27c0-4ab7-85b7-a5dbb9e9e2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:45.156527 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1c383-27c0-4ab7-85b7-a5dbb9e9e2fa None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:45.156806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1c383-27c0-4ab7-85b7-a5dbb9e9e2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:45.157130 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1c383-27c0-4ab7-85b7-a5dbb9e9e2fa None admin] PATH_INFO: `/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/a4a24d5a375945779fa626fae25ac237/roles/d820b1d3fa4144aca575c04645c099bc` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:45.157677 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 282/585] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:12:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:45.158509 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad19f2e-2f05-401d-b553-2c508f4f74d3 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:45.175565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1c383-27c0-4ab7-85b7-a5dbb9e9e2fa None admin] RBAC: Authorizing `identity:create_grant(project_id=cdae1cb71dd449b8a18062b7b52889e4, user_id=a4a24d5a375945779fa626fae25ac237, role_id=d820b1d3fa4144aca575c04645c099bc)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:45.177026 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1c383-27c0-4ab7-85b7-a5dbb9e9e2fa None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:45.242974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 304/586] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:12:45 2022] PUT /identity/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/a4a24d5a375945779fa626fae25ac237/roles/d820b1d3fa4144aca575c04645c099bc => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:12:45.243782 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1c383-27c0-4ab7-85b7-a5dbb9e9e2fa None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:45.255657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad19f2e-2f05-401d-b553-2c508f4f74d3 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:45.291083 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:45.297283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8207bc-0830-4a50-b705-1d0d295c3912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:45.298152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8207bc-0830-4a50-b705-1d0d295c3912 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:45.298467 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8207bc-0830-4a50-b705-1d0d295c3912 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:45.298793 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8207bc-0830-4a50-b705-1d0d295c3912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:45.311713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8207bc-0830-4a50-b705-1d0d295c3912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:45.313616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8207bc-0830-4a50-b705-1d0d295c3912 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:45.334796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-9a8207bc-0830-4a50-b705-1d0d295c3912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:45.334796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:45.334796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:45.334796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:45.334796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:45.334796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:45.334796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:45.334796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:45.334796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:45.334796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:45.334796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:45.334796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:45.334796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:45.334796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:45.334796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:45.335884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:45.335884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:45.335884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:45.335884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:45.335884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:45.335884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:45.335884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:45.335884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:45.335884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:45.335884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:45.335884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:45.335884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:45.335884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:45.335884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:45.335884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:45.335884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:45.335884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:45.336938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:45.336938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:45.336938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:45.336938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:45.336938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 283/587] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:45.337769 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8207bc-0830-4a50-b705-1d0d295c3912 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:45.410663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1c383-27c0-4ab7-85b7-a5dbb9e9e2fa None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:45.442250 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:45.454081 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eec960-59da-4f50-a1fb-a70ebc6407b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:45.459760 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eec960-59da-4f50-a1fb-a70ebc6407b9 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:45.459760 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eec960-59da-4f50-a1fb-a70ebc6407b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:45.459760 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eec960-59da-4f50-a1fb-a70ebc6407b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:45.459760 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eec960-59da-4f50-a1fb-a70ebc6407b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:45.459760 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eec960-59da-4f50-a1fb-a70ebc6407b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:45.459760 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eec960-59da-4f50-a1fb-a70ebc6407b9 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:45.470050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 305/588] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:45.474707 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eec960-59da-4f50-a1fb-a70ebc6407b9 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:45.488950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8207bc-0830-4a50-b705-1d0d295c3912 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:45.535878 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b979d-3911-428a-81f4-79c92d7fc53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:45.536678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b979d-3911-428a-81f4-79c92d7fc53f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:45.536958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b979d-3911-428a-81f4-79c92d7fc53f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:45.537227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b979d-3911-428a-81f4-79c92d7fc53f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:45.544554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b979d-3911-428a-81f4-79c92d7fc53f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:45.546003 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b979d-3911-428a-81f4-79c92d7fc53f None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:45.561409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-254b979d-3911-428a-81f4-79c92d7fc53f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 15:12:45.561409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:45.561409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:45.561409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:45.561409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:45.561409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:45.561409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:45.561409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:45.561409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:45.561409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:45.561409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:45.561409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:45.561409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:12:45.561409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:12:45.561409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:12:45.561409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:45.567790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:45.569168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:45.569168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:45.569168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:12:45.569168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:12:45.569168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:12:45.569168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:12:45.569168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:12:45.569168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:12:45.569168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 15:12:45.569168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:45.569168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 284/589] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:45.569168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b979d-3911-428a-81f4-79c92d7fc53f None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:45.591104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eec960-59da-4f50-a1fb-a70ebc6407b9 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:45.611644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:45.627618 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79b391-b86d-431e-a756-048463840382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:45.634286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79b391-b86d-431e-a756-048463840382 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:45.634286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79b391-b86d-431e-a756-048463840382 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:45.634286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79b391-b86d-431e-a756-048463840382 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:45.634286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79b391-b86d-431e-a756-048463840382 None admin] RBAC: Adding query filter params (role.id=d820b1d3fa4144aca575c04645c099bc, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=ed0d719c0eb04a3b8a228c6ce70fdf98) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:45.634286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79b391-b86d-431e-a756-048463840382 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:45.634286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79b391-b86d-431e-a756-048463840382 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:45.642601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 306/590] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:12:45 2022] GET /identity/v3/role_assignments?user.id=ed0d719c0eb04a3b8a228c6ce70fdf98&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=d820b1d3fa4144aca575c04645c099bc => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:45.642601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79b391-b86d-431e-a756-048463840382 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:45.686955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b979d-3911-428a-81f4-79c92d7fc53f None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:45.716266 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:45.732891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd40906-645e-49f1-b765-7bee87300b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:45.733563 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd40906-645e-49f1-b765-7bee87300b05 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:45.737304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd40906-645e-49f1-b765-7bee87300b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:45.737304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd40906-645e-49f1-b765-7bee87300b05 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:45.737304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd40906-645e-49f1-b765-7bee87300b05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:45.737304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd40906-645e-49f1-b765-7bee87300b05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:45.737304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd40906-645e-49f1-b765-7bee87300b05 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:45.750883 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 285/591] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:45.751822 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd40906-645e-49f1-b765-7bee87300b05 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:45.775639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79b391-b86d-431e-a756-048463840382 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:45.818299 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235dd5cd-12e8-497a-9f22-629706054f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:45.819240 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235dd5cd-12e8-497a-9f22-629706054f77 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:45.819529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235dd5cd-12e8-497a-9f22-629706054f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:45.819805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235dd5cd-12e8-497a-9f22-629706054f77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:45.827263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235dd5cd-12e8-497a-9f22-629706054f77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:45.828758 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235dd5cd-12e8-497a-9f22-629706054f77 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:45.845934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-235dd5cd-12e8-497a-9f22-629706054f77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:45.845934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:45.845934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:45.845934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:45.845934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:45.845934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:45.845934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:45.845934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:45.845934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:45.845934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:45.845934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:45.845934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:45.845934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:12:45.845934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:12:45.845934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:12:45.846921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:12:45.846921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:45.846921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:45.846921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:45.846921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:45.846921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:45.846921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:45.846921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:45.846921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:45.846921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:45.846921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:45.846921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:45.846921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:45.846921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:45.846921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:45.846921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:45.846921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:45.848364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:12:45.848364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:12:45.848364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:12:45.848364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:12:45.848364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:12:45.848364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:12:45.848364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:45.848364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:45.848364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 307/592] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:45.848908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235dd5cd-12e8-497a-9f22-629706054f77 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:45.886366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd40906-645e-49f1-b765-7bee87300b05 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:45.889956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:45.934974 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a954247-004d-4eb0-9bdb-55a2ccf73c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:45.936179 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a954247-004d-4eb0-9bdb-55a2ccf73c55 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:45.936623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a954247-004d-4eb0-9bdb-55a2ccf73c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:45.937257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a954247-004d-4eb0-9bdb-55a2ccf73c55 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:45.938748 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a954247-004d-4eb0-9bdb-55a2ccf73c55 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:45.939444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a954247-004d-4eb0-9bdb-55a2ccf73c55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:45.941673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a954247-004d-4eb0-9bdb-55a2ccf73c55 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:45.957224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235dd5cd-12e8-497a-9f22-629706054f77 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:45.966079 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 286/593] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Jul 20 15:12:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:45.967231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a954247-004d-4eb0-9bdb-55a2ccf73c55 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:45.968011 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:46.000300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e25f9-d870-4538-b693-8f6ec316a81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:46.002495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e25f9-d870-4538-b693-8f6ec316a81f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:46.003277 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e25f9-d870-4538-b693-8f6ec316a81f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:46.003898 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e25f9-d870-4538-b693-8f6ec316a81f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:46.014943 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85e25f9-d870-4538-b693-8f6ec316a81f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:46.016667 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85e25f9-d870-4538-b693-8f6ec316a81f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:46.038506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-d85e25f9-d870-4538-b693-8f6ec316a81f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 15:12:46.038506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:46.038506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:46.038506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:46.038506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:46.038506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:46.038506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:46.038506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:46.038506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:46.038506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:46.038506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:46.038506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:46.038506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:12:46.038506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:12:46.038506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:12:46.038506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:46.039676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:46.040826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:46.040826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:46.040826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:12:46.040826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:12:46.040826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:12:46.040826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:12:46.040826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:12:46.040826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:12:46.040826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 15:12:46.040826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:46.044010 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 308/594] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:46.045320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e25f9-d870-4538-b693-8f6ec316a81f None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:46.101214 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a954247-004d-4eb0-9bdb-55a2ccf73c55 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:46.146190 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12d082-c0a9-49e1-9ed4-8eeec0ab9885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:46.147580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12d082-c0a9-49e1-9ed4-8eeec0ab9885 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:46.148115 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12d082-c0a9-49e1-9ed4-8eeec0ab9885 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:46.148543 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12d082-c0a9-49e1-9ed4-8eeec0ab9885 None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:46.149988 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12d082-c0a9-49e1-9ed4-8eeec0ab9885 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:46.150620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12d082-c0a9-49e1-9ed4-8eeec0ab9885 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:46.152292 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12d082-c0a9-49e1-9ed4-8eeec0ab9885 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:46.152706 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e25f9-d870-4538-b693-8f6ec316a81f None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:46.173543 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 287/595] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:12:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:46.173543 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12d082-c0a9-49e1-9ed4-8eeec0ab9885 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:46.190634 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218828fe-579d-44b0-93ef-c17802b67d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:46.191615 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218828fe-579d-44b0-93ef-c17802b67d3b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:46.192033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218828fe-579d-44b0-93ef-c17802b67d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:46.192435 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218828fe-579d-44b0-93ef-c17802b67d3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:46.203352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218828fe-579d-44b0-93ef-c17802b67d3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:46.204876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218828fe-579d-44b0-93ef-c17802b67d3b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:46.205237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:46.224155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-218828fe-579d-44b0-93ef-c17802b67d3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:46.224155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:46.224155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:46.224155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:46.224155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:46.224155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:46.224155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:46.224155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:46.224155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:46.224155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:46.224155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:46.224155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:46.224155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:46.224155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:46.224155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:46.230774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:46.230774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:46.230774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:46.230774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:46.230774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:46.230774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:46.230774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:46.230774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:46.230774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:46.230774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:46.230774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:46.230774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:46.230774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:46.230774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:46.230774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:46.230774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:46.230774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:46.231953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:46.231953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:46.231953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:46.231953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:46.231953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 309/596] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:46.231953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218828fe-579d-44b0-93ef-c17802b67d3b None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:46.270664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:46.294179 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12d082-c0a9-49e1-9ed4-8eeec0ab9885 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:46.329329 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4111d-6209-40fe-8cbc-67d834c519ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:46.332452 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4111d-6209-40fe-8cbc-67d834c519ea None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:46.332452 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4111d-6209-40fe-8cbc-67d834c519ea None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:46.332452 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4111d-6209-40fe-8cbc-67d834c519ea None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:46.332452 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e4111d-6209-40fe-8cbc-67d834c519ea None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:46.332805 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e4111d-6209-40fe-8cbc-67d834c519ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:46.334396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e4111d-6209-40fe-8cbc-67d834c519ea None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:46.344780 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218828fe-579d-44b0-93ef-c17802b67d3b None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:46.355104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 288/597] 10.222.0.78 () {56 vars in 1139 bytes} [Wed Jul 20 15:12:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:46.356210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4111d-6209-40fe-8cbc-67d834c519ea None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:46.384239 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119d209-37ac-4910-9a88-da143060c27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:46.385456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119d209-37ac-4910-9a88-da143060c27f None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:46.386023 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119d209-37ac-4910-9a88-da143060c27f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:46.386474 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119d209-37ac-4910-9a88-da143060c27f None admin] PATH_INFO: `/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/b6c13596c2df438294bdac7c52ae8cb1/roles/d820b1d3fa4144aca575c04645c099bc` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:46.405815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119d209-37ac-4910-9a88-da143060c27f None admin] RBAC: Authorizing `identity:create_grant(project_id=cdae1cb71dd449b8a18062b7b52889e4, user_id=b6c13596c2df438294bdac7c52ae8cb1, role_id=d820b1d3fa4144aca575c04645c099bc)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:46.407529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119d209-37ac-4910-9a88-da143060c27f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:46.453731 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:46.473678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4111d-6209-40fe-8cbc-67d834c519ea None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:46.496886 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 310/598] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:12:46 2022] PUT /identity/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/b6c13596c2df438294bdac7c52ae8cb1/roles/d820b1d3fa4144aca575c04645c099bc => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:12:46.496886 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119d209-37ac-4910-9a88-da143060c27f None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:46.535862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5256bb-4f6b-48e8-85b2-a64ec70ad197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:46.538246 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5256bb-4f6b-48e8-85b2-a64ec70ad197 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:46.538556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5256bb-4f6b-48e8-85b2-a64ec70ad197 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:46.541987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5256bb-4f6b-48e8-85b2-a64ec70ad197 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:46.542869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5256bb-4f6b-48e8-85b2-a64ec70ad197 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:46.543191 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5256bb-4f6b-48e8-85b2-a64ec70ad197 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:46.544691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5256bb-4f6b-48e8-85b2-a64ec70ad197 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:46.572611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 289/599] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:46.580068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5256bb-4f6b-48e8-85b2-a64ec70ad197 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:46.651032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:46.696626 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119d209-37ac-4910-9a88-da143060c27f None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:46.739607 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5256bb-4f6b-48e8-85b2-a64ec70ad197 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:46.739965 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa040f-8e03-4b8c-ba17-50ba3fcd554b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:46.740805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa040f-8e03-4b8c-ba17-50ba3fcd554b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:46.741088 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa040f-8e03-4b8c-ba17-50ba3fcd554b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:46.741362 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa040f-8e03-4b8c-ba17-50ba3fcd554b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:46.753223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa040f-8e03-4b8c-ba17-50ba3fcd554b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:46.755242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa040f-8e03-4b8c-ba17-50ba3fcd554b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:46.773140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-1aaa040f-8e03-4b8c-ba17-50ba3fcd554b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:46.773140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:46.773140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:46.773140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:46.773140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:46.773140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:46.773140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:46.773140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:46.773140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:46.773140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:46.773140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:46.773140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:46.773140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:46.773140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:46.773140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:46.774106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:46.774106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:46.774106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:46.774106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:46.774106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:46.774106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:46.774106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:46.774106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:46.774106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:46.774106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:46.774106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:46.774106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:46.774106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:46.774106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:46.774106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:46.774106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:46.774106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:46.775058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:46.775058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:46.775058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:46.775058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:46.776058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 311/600] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:46.779888 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa040f-8e03-4b8c-ba17-50ba3fcd554b None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:46.781932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df826b10-1b49-40f9-be90-9ddc0592f6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:46.782991 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df826b10-1b49-40f9-be90-9ddc0592f6ac None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:46.783499 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df826b10-1b49-40f9-be90-9ddc0592f6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:46.783997 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df826b10-1b49-40f9-be90-9ddc0592f6ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:46.791560 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df826b10-1b49-40f9-be90-9ddc0592f6ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:46.793296 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df826b10-1b49-40f9-be90-9ddc0592f6ac None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:46.796208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:46.809903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-df826b10-1b49-40f9-be90-9ddc0592f6ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:46.809903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:46.809903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:46.809903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:46.809903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:46.809903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:46.809903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:46.809903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:46.809903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:46.809903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:46.809903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:46.809903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:46.809903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:12:46.809903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:12:46.809903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:12:46.816172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:12:46.816172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:46.816172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:46.816172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:46.816172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:46.816172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:46.816172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:46.816172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:46.816172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:46.816172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:46.816172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:46.816172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:46.816172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:46.816172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:46.816172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:46.816172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:46.816172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:46.817622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:12:46.817622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:12:46.817622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:12:46.817622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:12:46.817622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:12:46.817622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:12:46.817622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:46.817622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:46.817622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 290/601] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:46.823352 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df826b10-1b49-40f9-be90-9ddc0592f6ac None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:46.943860 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa040f-8e03-4b8c-ba17-50ba3fcd554b None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:46.967909 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:46.980769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336fb62-ab13-40ad-940a-5c92f9208ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:46.982230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336fb62-ab13-40ad-940a-5c92f9208ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:46.982230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336fb62-ab13-40ad-940a-5c92f9208ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:46.982773 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336fb62-ab13-40ad-940a-5c92f9208ef0 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:46.983732 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336fb62-ab13-40ad-940a-5c92f9208ef0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:46.984294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336fb62-ab13-40ad-940a-5c92f9208ef0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:46.985894 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336fb62-ab13-40ad-940a-5c92f9208ef0 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:47.000392 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 312/602] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:12:47.007481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336fb62-ab13-40ad-940a-5c92f9208ef0 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:47.023167 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df826b10-1b49-40f9-be90-9ddc0592f6ac None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:47.066303 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745bb5b-b38b-4710-90fb-d2203fb9ddde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:47.067474 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745bb5b-b38b-4710-90fb-d2203fb9ddde None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:47.068068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745bb5b-b38b-4710-90fb-d2203fb9ddde None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:47.068550 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745bb5b-b38b-4710-90fb-d2203fb9ddde None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:47.069565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745bb5b-b38b-4710-90fb-d2203fb9ddde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:47.070082 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745bb5b-b38b-4710-90fb-d2203fb9ddde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:47.071977 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745bb5b-b38b-4710-90fb-d2203fb9ddde None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:47.085157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 291/603] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:12:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:47.097241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745bb5b-b38b-4710-90fb-d2203fb9ddde None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:47.168631 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:47.217282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745bb5b-b38b-4710-90fb-d2203fb9ddde None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:47.244456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336fb62-ab13-40ad-940a-5c92f9208ef0 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:47.262042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee13933-a4e3-41bc-a15b-f9f8271f7230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:47.263365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee13933-a4e3-41bc-a15b-f9f8271f7230 None admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:47.264342 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee13933-a4e3-41bc-a15b-f9f8271f7230 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:47.264835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee13933-a4e3-41bc-a15b-f9f8271f7230 None admin] PATH_INFO: `/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/b51e9bdab4164d87b7b0b11c505c2c0a/roles/d820b1d3fa4144aca575c04645c099bc` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:47.284365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee13933-a4e3-41bc-a15b-f9f8271f7230 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdae1cb71dd449b8a18062b7b52889e4, user_id=b51e9bdab4164d87b7b0b11c505c2c0a, role_id=d820b1d3fa4144aca575c04645c099bc)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:47.286426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee13933-a4e3-41bc-a15b-f9f8271f7230 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:47.313568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653efc67-d075-425b-8be5-7f2a0a8575ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:47.314944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653efc67-d075-425b-8be5-7f2a0a8575ea None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:47.315507 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653efc67-d075-425b-8be5-7f2a0a8575ea None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:47.316029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653efc67-d075-425b-8be5-7f2a0a8575ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:47.325746 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653efc67-d075-425b-8be5-7f2a0a8575ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:47.327719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653efc67-d075-425b-8be5-7f2a0a8575ea None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:47.343880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-653efc67-d075-425b-8be5-7f2a0a8575ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:47.343880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:47.343880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:47.343880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:47.343880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:47.343880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:47.343880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:47.343880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:47.343880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:47.343880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:47.343880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:47.343880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:47.343880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:12:47.343880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:12:47.343880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:12:47.344867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:12:47.344867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:47.344867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:47.344867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:47.344867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:47.344867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:47.344867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:47.344867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:47.344867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:47.344867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:47.344867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:47.344867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:47.344867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:47.344867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:47.344867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:47.344867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:47.344867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:47.345831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:12:47.345831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:12:47.345831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:12:47.345831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:12:47.345831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:12:47.345831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:12:47.345831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:47.345831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:47.348762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 313/604] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:47.355014 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653efc67-d075-425b-8be5-7f2a0a8575ea None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:47.402757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 292/605] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:12:47 2022] PUT /identity/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/b51e9bdab4164d87b7b0b11c505c2c0a/roles/d820b1d3fa4144aca575c04645c099bc => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:12:47.404227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664aed63-e1df-4b3d-ae62-93b363fa7bfe None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:47.404527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664aed63-e1df-4b3d-ae62-93b363fa7bfe None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:47.404807 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664aed63-e1df-4b3d-ae62-93b363fa7bfe None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:47.406110 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:47.412330 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 293/606] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:47.418721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b26033-823b-4c2a-bdf9-02f890ea187d None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:47.419020 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b26033-823b-4c2a-bdf9-02f890ea187d None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:47.419294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b26033-823b-4c2a-bdf9-02f890ea187d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:47.458889 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-22b26033-823b-4c2a-bdf9-02f890ea187d None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:47.460200 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-22b26033-823b-4c2a-bdf9-02f890ea187d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:47.482302 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653efc67-d075-425b-8be5-7f2a0a8575ea None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:47.526354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46494d8d-c3e8-40d4-bd9b-59c45cf369fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:47.527135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46494d8d-c3e8-40d4-bd9b-59c45cf369fd None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:47.527724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46494d8d-c3e8-40d4-bd9b-59c45cf369fd None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:47.528014 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46494d8d-c3e8-40d4-bd9b-59c45cf369fd None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:47.528828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46494d8d-c3e8-40d4-bd9b-59c45cf369fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:47.529198 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46494d8d-c3e8-40d4-bd9b-59c45cf369fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:47.530907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46494d8d-c3e8-40d4-bd9b-59c45cf369fd None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:47.546629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 314/607] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:12:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:47.556406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46494d8d-c3e8-40d4-bd9b-59c45cf369fd None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:47.614655 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 294/608] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:47.665178 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b26033-823b-4c2a-bdf9-02f890ea187d None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:47.708117 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46494d8d-c3e8-40d4-bd9b-59c45cf369fd None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:47.718131 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:47.735844 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:47.758869 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7325c-0dd2-442d-8a54-fb4f86d6329a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:47.759775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7325c-0dd2-442d-8a54-fb4f86d6329a None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:47.760058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7325c-0dd2-442d-8a54-fb4f86d6329a None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:47.760336 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7325c-0dd2-442d-8a54-fb4f86d6329a None admin] PATH_INFO: `/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/b57530f18d9c4d328f8014db736d8de9/roles/d820b1d3fa4144aca575c04645c099bc` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:47.785825 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7325c-0dd2-442d-8a54-fb4f86d6329a None admin] RBAC: Authorizing `identity:create_grant(project_id=cdae1cb71dd449b8a18062b7b52889e4, user_id=b57530f18d9c4d328f8014db736d8de9, role_id=d820b1d3fa4144aca575c04645c099bc)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:47.787657 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7325c-0dd2-442d-8a54-fb4f86d6329a None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:47.821912 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b26033-823b-4c2a-bdf9-02f890ea187d None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:47.858623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be38b2b-4393-470c-99fb-037317ea7552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:47.859862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be38b2b-4393-470c-99fb-037317ea7552 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:47.860371 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be38b2b-4393-470c-99fb-037317ea7552 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:47.860651 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be38b2b-4393-470c-99fb-037317ea7552 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:47.874292 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 315/609] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:12:47 2022] PUT /identity/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/b57530f18d9c4d328f8014db736d8de9/roles/d820b1d3fa4144aca575c04645c099bc => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:12:47.875500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af00766-cecd-4f7e-bd02-534547839d89 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:47.875796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af00766-cecd-4f7e-bd02-534547839d89 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:47.876072 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af00766-cecd-4f7e-bd02-534547839d89 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:47.877566 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 316/610] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:47.884501 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7118b-dc2d-4066-9173-ccd45d6b12fa None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:47.884999 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7118b-dc2d-4066-9173-ccd45d6b12fa None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:47.885283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7118b-dc2d-4066-9173-ccd45d6b12fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:47.887338 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be38b2b-4393-470c-99fb-037317ea7552 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:47.895370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-1be38b2b-4393-470c-99fb-037317ea7552 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:47.895370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:47.895370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:47.895370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:47.895370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:47.895370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:47.895370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:47.895370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:47.895370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:47.895370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:47.895370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:47.895370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:47.895370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:47.895370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:47.895370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:47.895370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:47.903681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:47.903681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:47.903681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:47.903681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:47.903681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:47.903681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:47.903681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:47.903681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:47.903681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:47.903681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:47.903681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:47.903681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:47.903681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:47.903681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:47.903681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:47.903681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:47.904554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:47.904554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:47.904554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:47.904554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:47.904554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:47.904554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:47.904554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:47.904554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:47.904554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:47.904554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:47.904554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:47.904554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:47.904554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 295/611] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:47.916724 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be38b2b-4393-470c-99fb-037317ea7552 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:47.919546 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-43f7118b-dc2d-4066-9173-ccd45d6b12fa None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:47.921296 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-43f7118b-dc2d-4066-9173-ccd45d6b12fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:48.002209 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 317/612] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:48.043809 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7118b-dc2d-4066-9173-ccd45d6b12fa None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:48.093424 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be38b2b-4393-470c-99fb-037317ea7552 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:48.102608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:48.137352 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61deca-2fe9-41a4-afdd-4016db6170ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:48.138433 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61deca-2fe9-41a4-afdd-4016db6170ed None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:48.138978 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61deca-2fe9-41a4-afdd-4016db6170ed None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:48.139427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61deca-2fe9-41a4-afdd-4016db6170ed None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:48.140396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61deca-2fe9-41a4-afdd-4016db6170ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:48.140907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61deca-2fe9-41a4-afdd-4016db6170ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:48.142684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:48.143633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61deca-2fe9-41a4-afdd-4016db6170ed None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:48.161385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 296/613] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:12:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:12:48.167319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61deca-2fe9-41a4-afdd-4016db6170ed None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:48.181961 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7118b-dc2d-4066-9173-ccd45d6b12fa None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:48.248403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82867c-a65f-42ab-8db6-ba0ba828ef18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:48.249516 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82867c-a65f-42ab-8db6-ba0ba828ef18 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:48.249983 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82867c-a65f-42ab-8db6-ba0ba828ef18 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:48.250448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82867c-a65f-42ab-8db6-ba0ba828ef18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:48.271137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82867c-a65f-42ab-8db6-ba0ba828ef18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:48.280092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-0d82867c-a65f-42ab-8db6-ba0ba828ef18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:48.280092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:48.280092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:48.280092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:48.280092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:48.280092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:48.280092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:48.280092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:48.280092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:48.280092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:48.280092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:48.280092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:48.280092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:48.280092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:48.280092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:48.280092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:48.281047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:48.281047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:48.281047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:48.281047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:48.281047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:48.281047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:48.281047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:48.281047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:48.281047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:48.281047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:48.281047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:48.281047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:48.281047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:48.281047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:48.281047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:48.281047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:48.281947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:48.281947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:48.281947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:48.281947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:48.281947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:48.281947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:48.281947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:48.281947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:48.281947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:48.281947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:48.281947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:48.281947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:48.284632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 318/614] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:48.294576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82867c-a65f-42ab-8db6-ba0ba828ef18 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:48.346458 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61deca-2fe9-41a4-afdd-4016db6170ed None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:48.412660 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ad99d-bd47-49b8-986a-095c4be7fd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:48.413722 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ad99d-bd47-49b8-986a-095c4be7fd96 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:48.414187 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ad99d-bd47-49b8-986a-095c4be7fd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:48.414715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ad99d-bd47-49b8-986a-095c4be7fd96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:48.429049 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ad99d-bd47-49b8-986a-095c4be7fd96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:48.435838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ad99d-bd47-49b8-986a-095c4be7fd96 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:48.452112 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82867c-a65f-42ab-8db6-ba0ba828ef18 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:48.457794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-c05ad99d-bd47-49b8-986a-095c4be7fd96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:48.457794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:48.457794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:48.457794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:48.457794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:48.457794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:48.457794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:48.457794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:48.457794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:48.457794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:48.457794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:48.457794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:48.457794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:48.457794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:48.457794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:48.458919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:48.458919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:48.458919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:48.458919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:48.458919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:48.458919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:48.458919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:48.458919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:48.458919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:48.458919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:48.458919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:48.458919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:48.458919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:48.458919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:48.458919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:48.458919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:48.458919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:48.460057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:48.460057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:48.460057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:48.460057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:48.460057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 297/615] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:48.476444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ad99d-bd47-49b8-986a-095c4be7fd96 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:48.497956 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4dd356-1a94-4b0c-8e6d-6d42da7d8c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:48.498925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4dd356-1a94-4b0c-8e6d-6d42da7d8c40 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:48.499365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4dd356-1a94-4b0c-8e6d-6d42da7d8c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:48.499793 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4dd356-1a94-4b0c-8e6d-6d42da7d8c40 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:48.501072 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4dd356-1a94-4b0c-8e6d-6d42da7d8c40 None admin] RBAC: Adding query filter params (name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:48.501872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4dd356-1a94-4b0c-8e6d-6d42da7d8c40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:48.503759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4dd356-1a94-4b0c-8e6d-6d42da7d8c40 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:48.513310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 319/616] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:12:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:48.518613 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4dd356-1a94-4b0c-8e6d-6d42da7d8c40 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:48.520413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:48.524657 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:48.633316 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4dd356-1a94-4b0c-8e6d-6d42da7d8c40 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:48.670862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f50a5-655b-4f4c-be57-344353b5d2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:48.672107 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f50a5-655b-4f4c-be57-344353b5d2cf None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:48.672606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f50a5-655b-4f4c-be57-344353b5d2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:48.673034 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f50a5-655b-4f4c-be57-344353b5d2cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:48.685000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f50a5-655b-4f4c-be57-344353b5d2cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:48.686708 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f50a5-655b-4f4c-be57-344353b5d2cf None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:48.705590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-0c5f50a5-655b-4f4c-be57-344353b5d2cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:48.705590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:48.705590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:48.705590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:48.705590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:48.705590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:48.705590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:48.705590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:48.705590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:48.705590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:48.705590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:48.705590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:48.705590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:48.705590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:48.705590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:48.706763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:48.706763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:48.706763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:48.706763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:48.706763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:48.706763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:48.706763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:48.706763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:48.706763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:48.706763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:48.706763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:48.706763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:48.706763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:48.706763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:48.706763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:48.706763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:48.706763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:48.708657 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:48.708657 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:48.708657 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:48.708657 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:48.710655 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 320/617] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:48.718748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f50a5-655b-4f4c-be57-344353b5d2cf None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:48.774201 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ad99d-bd47-49b8-986a-095c4be7fd96 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:48.827223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:48.834913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bf9bb-4e81-47d9-b495-41c921c6a40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:48.839696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bf9bb-4e81-47d9-b495-41c921c6a40b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:48.839696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bf9bb-4e81-47d9-b495-41c921c6a40b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:48.839696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bf9bb-4e81-47d9-b495-41c921c6a40b None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:48.839696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bf9bb-4e81-47d9-b495-41c921c6a40b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:48.839696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bf9bb-4e81-47d9-b495-41c921c6a40b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:48.839696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bf9bb-4e81-47d9-b495-41c921c6a40b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:48.851385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 298/618] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:48.856655 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bf9bb-4e81-47d9-b495-41c921c6a40b None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:48.921970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f50a5-655b-4f4c-be57-344353b5d2cf None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:48.958819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e6aa6-a000-4b11-b3ff-94366c375667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:48.959589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e6aa6-a000-4b11-b3ff-94366c375667 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:48.959873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e6aa6-a000-4b11-b3ff-94366c375667 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:48.960149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e6aa6-a000-4b11-b3ff-94366c375667 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:48.960928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e6aa6-a000-4b11-b3ff-94366c375667 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:48.961238 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e6aa6-a000-4b11-b3ff-94366c375667 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:48.962631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e6aa6-a000-4b11-b3ff-94366c375667 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:48.980242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 321/619] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:48.981436 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e6aa6-a000-4b11-b3ff-94366c375667 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:49.000272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:49.032924 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bf9bb-4e81-47d9-b495-41c921c6a40b None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:49.078028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276cdba-6862-4b27-82d3-e278674c10da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:49.078884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276cdba-6862-4b27-82d3-e278674c10da None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:49.079168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276cdba-6862-4b27-82d3-e278674c10da None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:49.079606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276cdba-6862-4b27-82d3-e278674c10da None admin] PATH_INFO: `/v3/users/nova` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:49.089222 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276cdba-6862-4b27-82d3-e278674c10da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:49.090723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276cdba-6862-4b27-82d3-e278674c10da None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:49.107412 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e6aa6-a000-4b11-b3ff-94366c375667 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:49.107628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-7276cdba-6862-4b27-82d3-e278674c10da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:12:49.107628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:49.107628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:49.107628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:49.107628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:49.107628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:49.107628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:49.107628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:49.107628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:49.107628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:49.107628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:49.107628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:49.107628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:12:49.107628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:12:49.107628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:12:49.107628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:49.108757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:49.109782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:49.109782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:49.109782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:12:49.109782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:12:49.109782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:12:49.109782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:12:49.109782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:12:49.109782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:12:49.109782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:12:49.109782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:49.109782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 299/620] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Jul 20 15:12:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:49.111425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00a9e6-4178-4d98-bae8-fff3f90c6b58 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:49.111425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00a9e6-4178-4d98-bae8-fff3f90c6b58 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:49.111425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00a9e6-4178-4d98-bae8-fff3f90c6b58 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:49.116979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 300/621] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:49.116979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00a9e6-4178-4d98-bae8-fff3f90c6b58 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:49.134812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:49.152631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03bb6a-bce5-414b-be70-5804e408947b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:49.153516 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03bb6a-bce5-414b-be70-5804e408947b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:49.153788 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03bb6a-bce5-414b-be70-5804e408947b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:49.154066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03bb6a-bce5-414b-be70-5804e408947b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:49.163669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03bb6a-bce5-414b-be70-5804e408947b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:49.166088 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03bb6a-bce5-414b-be70-5804e408947b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:49.190579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-eb03bb6a-bce5-414b-be70-5804e408947b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:12:49.190579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:49.190579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:49.190579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:49.190579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:49.190579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:49.190579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:49.190579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:49.190579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:49.190579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:49.190579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:49.190579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:49.190579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:12:49.190579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:12:49.190579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:12:49.190579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:49.192246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:49.193232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:49.193232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:49.193232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:12:49.193232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:12:49.193232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:12:49.193232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:12:49.193232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:12:49.193232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:12:49.193232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:12:49.193232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:49.197129 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 322/622] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:12:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:49.198771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c909265-c6ad-4e31-b2ff-87ba2e950628 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:49.199301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c909265-c6ad-4e31-b2ff-87ba2e950628 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:49.199799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c909265-c6ad-4e31-b2ff-87ba2e950628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:49.217260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-4c909265-c6ad-4e31-b2ff-87ba2e950628 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:49.218830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-4c909265-c6ad-4e31-b2ff-87ba2e950628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:49.253721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00a9e6-4178-4d98-bae8-fff3f90c6b58 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:49.289715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952ca4e-0da1-48e1-a0fc-c887fe449644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:49.290408 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952ca4e-0da1-48e1-a0fc-c887fe449644 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:49.290690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952ca4e-0da1-48e1-a0fc-c887fe449644 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:49.290962 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952ca4e-0da1-48e1-a0fc-c887fe449644 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:49.291683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 323/623] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:49.291911 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952ca4e-0da1-48e1-a0fc-c887fe449644 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:49.292221 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952ca4e-0da1-48e1-a0fc-c887fe449644 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:49.293056 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c909265-c6ad-4e31-b2ff-87ba2e950628 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:49.293761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952ca4e-0da1-48e1-a0fc-c887fe449644 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:49.295511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:49.313532 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 301/624] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Jul 20 15:12:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:49.319829 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952ca4e-0da1-48e1-a0fc-c887fe449644 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:49.429819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c909265-c6ad-4e31-b2ff-87ba2e950628 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:49.440212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:49.456788 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952ca4e-0da1-48e1-a0fc-c887fe449644 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:49.476468 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaca79a-ab7e-4a06-b250-71cd211f74e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:49.477986 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaca79a-ab7e-4a06-b250-71cd211f74e4 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:49.478481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaca79a-ab7e-4a06-b250-71cd211f74e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:49.478926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaca79a-ab7e-4a06-b250-71cd211f74e4 None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:49.479973 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaca79a-ab7e-4a06-b250-71cd211f74e4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:49.480439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaca79a-ab7e-4a06-b250-71cd211f74e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:49.482984 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaca79a-ab7e-4a06-b250-71cd211f74e4 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:49.496846 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b0c2c-b1c3-41a2-925d-edd6bf59114d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:49.497630 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b0c2c-b1c3-41a2-925d-edd6bf59114d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:49.498021 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b0c2c-b1c3-41a2-925d-edd6bf59114d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:49.498449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b0c2c-b1c3-41a2-925d-edd6bf59114d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:49.506895 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 324/625] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Jul 20 15:12:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:49.507963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaca79a-ab7e-4a06-b250-71cd211f74e4 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:49.571064 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:49.585153 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b0c2c-b1c3-41a2-925d-edd6bf59114d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:49.586998 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b0c2c-b1c3-41a2-925d-edd6bf59114d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:49.622021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaca79a-ab7e-4a06-b250-71cd211f74e4 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:49.658955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e65bb-4f9a-4a16-82ec-89b1d5628142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:49.660070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e65bb-4f9a-4a16-82ec-89b1d5628142 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:49.660482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e65bb-4f9a-4a16-82ec-89b1d5628142 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:49.660858 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e65bb-4f9a-4a16-82ec-89b1d5628142 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:49.671892 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e65bb-4f9a-4a16-82ec-89b1d5628142 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:49.680237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-065e65bb-4f9a-4a16-82ec-89b1d5628142 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:49.680237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:49.680237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:49.680237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:49.680237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:49.680237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:49.680237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:49.680237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:49.680237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:49.680237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:49.680237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:49.680237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:49.680237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:49.680237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:49.680237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:49.680237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:49.681343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:49.681343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:49.681343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:49.681343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:49.681343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:49.681343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:49.681343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:49.681343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:49.681343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:49.681343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:49.681343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:49.681343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:49.681343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:49.681343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:49.681343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:49.681343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:49.682420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:49.682420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:49.682420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:49.682420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:49.682420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:49.682420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:49.682420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:49.682420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:49.682420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:49.682420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:49.682420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:49.682420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:49.689047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 325/626] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:49.690093 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e65bb-4f9a-4a16-82ec-89b1d5628142 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:49.700236 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 302/627] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 15:12:49.705479 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b0c2c-b1c3-41a2-925d-edd6bf59114d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:49.718077 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:49.798739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e65bb-4f9a-4a16-82ec-89b1d5628142 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:49.802564 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:49.812527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b0c2c-b1c3-41a2-925d-edd6bf59114d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:49.836394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877fe20-c393-4d52-9c17-a21684e5a56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:49.837395 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877fe20-c393-4d52-9c17-a21684e5a56d None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:49.837807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877fe20-c393-4d52-9c17-a21684e5a56d None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:49.838325 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877fe20-c393-4d52-9c17-a21684e5a56d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:49.851324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6794d-bd8f-43f1-82d6-5a0ff7ab071f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:49.852118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877fe20-c393-4d52-9c17-a21684e5a56d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:49.852401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6794d-bd8f-43f1-82d6-5a0ff7ab071f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:49.852843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6794d-bd8f-43f1-82d6-5a0ff7ab071f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:49.853262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6794d-bd8f-43f1-82d6-5a0ff7ab071f None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:49.853978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877fe20-c393-4d52-9c17-a21684e5a56d None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:49.854433 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6794d-bd8f-43f1-82d6-5a0ff7ab071f None admin] RBAC: Adding query filter params (name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:49.854924 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6794d-bd8f-43f1-82d6-5a0ff7ab071f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:49.856550 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6794d-bd8f-43f1-82d6-5a0ff7ab071f None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:49.865689 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 303/628] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:12:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:49.866773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6794d-bd8f-43f1-82d6-5a0ff7ab071f None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:49.878907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-9877fe20-c393-4d52-9c17-a21684e5a56d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:49.878907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:49.878907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:49.878907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:49.878907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:49.878907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:49.878907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:49.878907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:49.878907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:49.878907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:49.878907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:49.878907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:49.878907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:49.878907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:49.878907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:49.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:49.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:49.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:49.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:49.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:49.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:49.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:49.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:49.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:49.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:49.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:49.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:49.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:49.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:49.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:49.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:49.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:49.880796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:49.880796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:49.880796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:49.880796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:49.880796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 326/629] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:49.882412 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877fe20-c393-4d52-9c17-a21684e5a56d None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:49.962785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:49.979140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6794d-bd8f-43f1-82d6-5a0ff7ab071f None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:49.987575 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877fe20-c393-4d52-9c17-a21684e5a56d None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:50.015695 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466935c-f925-4eb0-8131-9dd25c0d6d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:50.016488 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466935c-f925-4eb0-8131-9dd25c0d6d61 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:50.016766 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466935c-f925-4eb0-8131-9dd25c0d6d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:50.017041 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466935c-f925-4eb0-8131-9dd25c0d6d61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:50.026814 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178c0a5-bd4a-480f-9d06-7d0716179f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:50.027876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c0a5-bd4a-480f-9d06-7d0716179f52 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:50.028157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f466935c-f925-4eb0-8131-9dd25c0d6d61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:50.028633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c0a5-bd4a-480f-9d06-7d0716179f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:50.029130 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c0a5-bd4a-480f-9d06-7d0716179f52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:50.029591 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f466935c-f925-4eb0-8131-9dd25c0d6d61 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:50.042519 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178c0a5-bd4a-480f-9d06-7d0716179f52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:50.044232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178c0a5-bd4a-480f-9d06-7d0716179f52 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:50.047317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-f466935c-f925-4eb0-8131-9dd25c0d6d61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:50.047317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:50.047317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:50.047317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:50.047317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:50.047317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:50.047317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:50.047317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:50.047317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:50.047317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:50.047317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:50.047317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:50.047317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:50.047317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:50.047317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:50.048213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:50.048213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:50.048213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:50.048213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:50.048213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:50.048213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:50.048213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:50.048213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:50.048213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:50.048213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:50.048213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:50.048213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:50.048213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:50.048213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:50.048213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:50.048213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:50.048213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:50.049136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:50.049136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:50.049136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:50.049136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:50.049136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 304/630] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:50.049420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466935c-f925-4eb0-8131-9dd25c0d6d61 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:50.062466 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:50.064807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-a178c0a5-bd4a-480f-9d06-7d0716179f52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:50.064807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:50.064807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:50.064807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:50.064807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:50.064807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:50.064807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:50.064807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:50.064807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:50.064807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:50.064807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:50.064807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:50.064807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:50.064807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:50.064807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:50.065683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:50.065683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:50.065683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:50.065683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:50.065683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:50.065683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:50.065683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:50.065683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:50.065683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:50.065683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:50.065683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:50.065683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:50.065683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:50.065683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:50.065683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:50.065683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:50.065683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:50.067638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:50.067638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:50.067638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:50.067638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:50.067638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 327/631] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:50.067638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87830c-38f0-4116-b476-c445454cc4a5 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:50.067638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87830c-38f0-4116-b476-c445454cc4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:50.068036 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87830c-38f0-4116-b476-c445454cc4a5 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:50.073146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 328/632] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:50.073146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87830c-38f0-4116-b476-c445454cc4a5 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:50.171779 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466935c-f925-4eb0-8131-9dd25c0d6d61 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:50.176786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87830c-38f0-4116-b476-c445454cc4a5 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:50.202290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:50.204123 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6ee09-b4a4-4ea4-8f66-558b41074a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:50.204783 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6ee09-b4a4-4ea4-8f66-558b41074a2d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:50.205069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6ee09-b4a4-4ea4-8f66-558b41074a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:50.205347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6ee09-b4a4-4ea4-8f66-558b41074a2d None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:50.206067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6ee09-b4a4-4ea4-8f66-558b41074a2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:50.206403 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6ee09-b4a4-4ea4-8f66-558b41074a2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:50.208358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6ee09-b4a4-4ea4-8f66-558b41074a2d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:50.210548 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b8a7a-d68f-45ca-b36a-5c305171658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:50.216488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b8a7a-d68f-45ca-b36a-5c305171658c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:50.216488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b8a7a-d68f-45ca-b36a-5c305171658c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:50.216488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b8a7a-d68f-45ca-b36a-5c305171658c None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:50.216488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b8a7a-d68f-45ca-b36a-5c305171658c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:50.216488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b8a7a-d68f-45ca-b36a-5c305171658c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:50.216488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b8a7a-d68f-45ca-b36a-5c305171658c None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:50.221486 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 305/633] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:50.222430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6ee09-b4a4-4ea4-8f66-558b41074a2d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:50.225240 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 329/634] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:50.226738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20d18b-47da-4fa2-a56a-a54e55b2fadb None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:50.226992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20d18b-47da-4fa2-a56a-a54e55b2fadb None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:50.227281 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20d18b-47da-4fa2-a56a-a54e55b2fadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:50.243984 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-af20d18b-47da-4fa2-a56a-a54e55b2fadb None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:50.245180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-af20d18b-47da-4fa2-a56a-a54e55b2fadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:50.289159 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:50.309441 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 330/635] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:50.310840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75281876-fdb6-4a8b-ad5a-501dd23a07d3 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:50.311125 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75281876-fdb6-4a8b-ad5a-501dd23a07d3 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:50.311404 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75281876-fdb6-4a8b-ad5a-501dd23a07d3 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:50.312796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 331/636] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:50.317960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75281876-fdb6-4a8b-ad5a-501dd23a07d3 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:50.322973 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6ee09-b4a4-4ea4-8f66-558b41074a2d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:50.358376 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c4932-d9c5-433a-9309-681ad3e7e651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:50.359039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c4932-d9c5-433a-9309-681ad3e7e651 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:50.359326 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c4932-d9c5-433a-9309-681ad3e7e651 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:50.359604 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c4932-d9c5-433a-9309-681ad3e7e651 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:50.360326 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c4932-d9c5-433a-9309-681ad3e7e651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:50.360674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c4932-d9c5-433a-9309-681ad3e7e651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:50.362069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c4932-d9c5-433a-9309-681ad3e7e651 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:50.374595 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 306/637] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:50.375376 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c4932-d9c5-433a-9309-681ad3e7e651 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:50.423978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75281876-fdb6-4a8b-ad5a-501dd23a07d3 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:50.440310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:50.459647 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c93c49-c272-429e-9510-7bb0ef2cc289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:50.460440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c93c49-c272-429e-9510-7bb0ef2cc289 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:50.460722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c93c49-c272-429e-9510-7bb0ef2cc289 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:50.461001 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c93c49-c272-429e-9510-7bb0ef2cc289 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:50.467798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c93c49-c272-429e-9510-7bb0ef2cc289 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:50.469181 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c93c49-c272-429e-9510-7bb0ef2cc289 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:50.472854 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c4932-d9c5-433a-9309-681ad3e7e651 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:50.483955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-c0c93c49-c272-429e-9510-7bb0ef2cc289 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:50.483955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:50.483955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:50.483955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:50.483955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:50.483955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:50.483955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:50.483955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:50.483955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:50.483955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:50.483955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:50.483955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:50.483955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:12:50.483955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:12:50.483955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:12:50.484914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:12:50.484914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:50.484914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:50.484914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:50.484914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:50.484914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:50.484914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:50.484914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:50.484914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:50.484914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:50.484914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:50.484914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:50.484914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:50.484914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:50.484914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:50.484914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:50.484914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:50.485840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:12:50.485840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:12:50.485840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:12:50.485840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:12:50.485840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:12:50.485840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:12:50.485840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:50.485840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:50.485840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 332/638] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:50.486454 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866c389-4dbc-4273-ba44-fdcca14128b3 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:50.486771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866c389-4dbc-4273-ba44-fdcca14128b3 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:50.487039 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866c389-4dbc-4273-ba44-fdcca14128b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:50.502462 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:50.503418 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-a866c389-4dbc-4273-ba44-fdcca14128b3 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:50.505186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-a866c389-4dbc-4273-ba44-fdcca14128b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:50.511584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b6f6d-c074-4823-9583-6ee1247c0c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:50.512512 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b6f6d-c074-4823-9583-6ee1247c0c08 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:50.512792 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b6f6d-c074-4823-9583-6ee1247c0c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:50.513061 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b6f6d-c074-4823-9583-6ee1247c0c08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:50.520389 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b6f6d-c074-4823-9583-6ee1247c0c08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:50.521915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b6f6d-c074-4823-9583-6ee1247c0c08 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:50.536568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-de6b6f6d-c074-4823-9583-6ee1247c0c08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:50.536568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:50.536568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:50.536568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:50.536568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:50.536568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:50.536568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:50.536568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:50.536568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:50.536568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:50.536568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:50.536568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:50.536568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:12:50.536568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:12:50.536568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:12:50.537645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:12:50.537645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:50.537645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:50.537645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:50.537645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:50.537645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:50.537645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:50.537645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:50.537645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:50.537645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:50.537645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:50.537645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:50.537645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:50.537645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:50.537645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:50.537645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:50.537645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:50.538960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:12:50.538960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:12:50.538960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:12:50.538960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:12:50.538960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:12:50.538960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:12:50.538960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:50.538960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:50.538960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 307/639] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:50.538960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b6f6d-c074-4823-9583-6ee1247c0c08 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:50.575125 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 333/640] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:50.575974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866c389-4dbc-4273-ba44-fdcca14128b3 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:50.643334 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b6f6d-c074-4823-9583-6ee1247c0c08 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:50.676593 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a2e89-28c0-4598-be04-a3ac1c98f2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:50.677384 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a2e89-28c0-4598-be04-a3ac1c98f2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:50.677663 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a2e89-28c0-4598-be04-a3ac1c98f2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:50.677938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a2e89-28c0-4598-be04-a3ac1c98f2f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:50.681567 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866c389-4dbc-4273-ba44-fdcca14128b3 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:50.687922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a2e89-28c0-4598-be04-a3ac1c98f2f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:50.690605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:50.696262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-1a4a2e89-28c0-4598-be04-a3ac1c98f2f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:50.696262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:50.696262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:50.696262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:50.696262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:50.696262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:50.696262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:50.696262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:50.696262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:50.696262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:50.696262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:50.696262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:50.696262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:50.696262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:50.696262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:50.696262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:50.702530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:50.702530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:50.702530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:50.702530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:50.702530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:50.702530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:50.702530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:50.702530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:50.702530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:50.702530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:50.702530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:50.702530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:50.702530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:50.702530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:50.702530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:50.702530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:50.703435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:50.703435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:50.703435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:50.703435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:50.703435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:50.703435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:50.703435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:50.703435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:50.703435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:50.703435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:50.703435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:50.703435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:50.703435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 308/641] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:50.703435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a2e89-28c0-4598-be04-a3ac1c98f2f6 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:50.715206 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a944a3-1ed5-4755-bebf-055ef065aa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:50.716031 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a944a3-1ed5-4755-bebf-055ef065aa53 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:50.716316 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a944a3-1ed5-4755-bebf-055ef065aa53 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:50.716670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a944a3-1ed5-4755-bebf-055ef065aa53 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:50.724508 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a944a3-1ed5-4755-bebf-055ef065aa53 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:50.727060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a944a3-1ed5-4755-bebf-055ef065aa53 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:50.741724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-51a944a3-1ed5-4755-bebf-055ef065aa53 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 15:12:50.741724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:50.741724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:50.741724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:50.741724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:50.741724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:50.741724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:50.741724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:50.741724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:50.741724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:50.741724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:50.741724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:50.741724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:12:50.741724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:12:50.741724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:12:50.741724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:50.742830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:50.743872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:50.743872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:50.743872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:12:50.743872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:12:50.743872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:12:50.743872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:12:50.743872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:12:50.743872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:12:50.743872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 15:12:50.743872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:50.743872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 334/642] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:50.743872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a944a3-1ed5-4755-bebf-055ef065aa53 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:50.748569 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:50.802508 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a2e89-28c0-4598-be04-a3ac1c98f2f6 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:50.834987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8cccb2-522a-4cd2-adca-8c3ee79d682d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:50.835634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cccb2-522a-4cd2-adca-8c3ee79d682d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:50.835914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cccb2-522a-4cd2-adca-8c3ee79d682d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:50.836183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cccb2-522a-4cd2-adca-8c3ee79d682d None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:50.836965 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8cccb2-522a-4cd2-adca-8c3ee79d682d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:50.837266 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8cccb2-522a-4cd2-adca-8c3ee79d682d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:50.838699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8cccb2-522a-4cd2-adca-8c3ee79d682d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:50.842916 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a944a3-1ed5-4755-bebf-055ef065aa53 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:50.850490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 309/643] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:12:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:50.851364 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8cccb2-522a-4cd2-adca-8c3ee79d682d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:50.876800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d61ddc-7895-4201-b9ab-20c9c818f6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:50.877458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d61ddc-7895-4201-b9ab-20c9c818f6ca None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:50.878004 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d61ddc-7895-4201-b9ab-20c9c818f6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:50.878301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d61ddc-7895-4201-b9ab-20c9c818f6ca None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:50.879686 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d61ddc-7895-4201-b9ab-20c9c818f6ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:50.879990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d61ddc-7895-4201-b9ab-20c9c818f6ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:50.881416 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d61ddc-7895-4201-b9ab-20c9c818f6ca None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:50.893031 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 335/644] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:12:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:50.893808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d61ddc-7895-4201-b9ab-20c9c818f6ca None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:50.921042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:50.953672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8cccb2-522a-4cd2-adca-8c3ee79d682d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:50.961389 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:50.986970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd30263-4136-4fbb-a824-f955dc03e122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:50.987703 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd30263-4136-4fbb-a824-f955dc03e122 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:50.988573 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd30263-4136-4fbb-a824-f955dc03e122 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:50.988856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd30263-4136-4fbb-a824-f955dc03e122 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:50.991823 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d61ddc-7895-4201-b9ab-20c9c818f6ca None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:50.998871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd30263-4136-4fbb-a824-f955dc03e122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:51.007237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-dbd30263-4136-4fbb-a824-f955dc03e122 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:51.007237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:51.007237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:51.007237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:51.007237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:51.007237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:51.007237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:51.007237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:51.007237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:51.007237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:51.007237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:51.007237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:51.007237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:51.007237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:51.007237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:51.007237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:51.008360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:51.008360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:51.008360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:51.008360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:51.008360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:51.008360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:51.008360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:51.008360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:51.008360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:51.008360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:51.008360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:51.008360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:51.008360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:51.008360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:51.008360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:51.008360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:51.009319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:51.009319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:51.009319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:51.009319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:51.009319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:51.009319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:51.009319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:51.009319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:51.009319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:51.009319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:51.009319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:51.009319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:51.009319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 310/645] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:51.009319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd30263-4136-4fbb-a824-f955dc03e122 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:51.027379 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f3450-1250-4070-bf90-b57418d279f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:51.028045 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f3450-1250-4070-bf90-b57418d279f0 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:51.028331 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f3450-1250-4070-bf90-b57418d279f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:51.028603 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f3450-1250-4070-bf90-b57418d279f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:51.029339 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f3450-1250-4070-bf90-b57418d279f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:51.029644 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f3450-1250-4070-bf90-b57418d279f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:51.031034 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f3450-1250-4070-bf90-b57418d279f0 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:51.039338 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 336/646] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:12:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:51.040070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f3450-1250-4070-bf90-b57418d279f0 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:51.117674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd30263-4136-4fbb-a824-f955dc03e122 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:51.131319 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:51.140047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f3450-1250-4070-bf90-b57418d279f0 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:51.150818 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e31155-b819-47c4-9844-9cd86656f788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:51.151487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e31155-b819-47c4-9844-9cd86656f788 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:51.151769 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e31155-b819-47c4-9844-9cd86656f788 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:51.152047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e31155-b819-47c4-9844-9cd86656f788 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:51.152885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e31155-b819-47c4-9844-9cd86656f788 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:51.153195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e31155-b819-47c4-9844-9cd86656f788 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:51.154645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e31155-b819-47c4-9844-9cd86656f788 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:51.172247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 311/647] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Jul 20 15:12:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:51.173034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e31155-b819-47c4-9844-9cd86656f788 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:51.174608 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa2340-898b-487e-aee6-d9bbc7e0ce20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:51.176106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa2340-898b-487e-aee6-d9bbc7e0ce20 None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:51.176437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa2340-898b-487e-aee6-d9bbc7e0ce20 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:51.176713 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa2340-898b-487e-aee6-d9bbc7e0ce20 None admin] PATH_INFO: `/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/ed0d719c0eb04a3b8a228c6ce70fdf98/roles/d820b1d3fa4144aca575c04645c099bc` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:51.190880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa2340-898b-487e-aee6-d9bbc7e0ce20 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdae1cb71dd449b8a18062b7b52889e4, user_id=ed0d719c0eb04a3b8a228c6ce70fdf98, role_id=d820b1d3fa4144aca575c04645c099bc)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:51.192428 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa2340-898b-487e-aee6-d9bbc7e0ce20 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:51.198424 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:51.256290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 337/648] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:12:51 2022] PUT /identity/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/ed0d719c0eb04a3b8a228c6ce70fdf98/roles/d820b1d3fa4144aca575c04645c099bc => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:12:51.257385 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa2340-898b-487e-aee6-d9bbc7e0ce20 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:51.283265 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e31155-b819-47c4-9844-9cd86656f788 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:51.325736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8aefd-f842-4022-8789-8eecdecc4ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:51.326454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8aefd-f842-4022-8789-8eecdecc4ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:51.326739 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8aefd-f842-4022-8789-8eecdecc4ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:51.327016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8aefd-f842-4022-8789-8eecdecc4ab6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:51.328493 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8aefd-f842-4022-8789-8eecdecc4ab6 None admin] RBAC: Adding query filter params (role.id=d820b1d3fa4144aca575c04645c099bc, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=a4a24d5a375945779fa626fae25ac237) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:51.328811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8aefd-f842-4022-8789-8eecdecc4ab6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:51.330254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8aefd-f842-4022-8789-8eecdecc4ab6 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:51.346294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 312/649] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:12:51 2022] GET /identity/v3/role_assignments?user.id=a4a24d5a375945779fa626fae25ac237&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=d820b1d3fa4144aca575c04645c099bc => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:51.346294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8aefd-f842-4022-8789-8eecdecc4ab6 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:51.398951 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa2340-898b-487e-aee6-d9bbc7e0ce20 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:51.448820 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c16ae-0677-4a26-bee1-87799f98f2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:51.449620 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c16ae-0677-4a26-bee1-87799f98f2ab None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:51.449901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c16ae-0677-4a26-bee1-87799f98f2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:51.450174 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c16ae-0677-4a26-bee1-87799f98f2ab None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:51.450972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c16ae-0677-4a26-bee1-87799f98f2ab None admin] RBAC: Adding query filter params (name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:51.451277 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c16ae-0677-4a26-bee1-87799f98f2ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:51.452684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c16ae-0677-4a26-bee1-87799f98f2ab None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:51.464230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 338/650] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:12:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:51.465219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c16ae-0677-4a26-bee1-87799f98f2ab None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:51.470781 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:51.470982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:51.485544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8aefd-f842-4022-8789-8eecdecc4ab6 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:51.534182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a108c-4c3d-4617-aa48-bd1ab5aef9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:51.535793 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a108c-4c3d-4617-aa48-bd1ab5aef9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:51.536070 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a108c-4c3d-4617-aa48-bd1ab5aef9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:51.536341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a108c-4c3d-4617-aa48-bd1ab5aef9f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:51.548594 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a108c-4c3d-4617-aa48-bd1ab5aef9f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:51.550040 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a108c-4c3d-4617-aa48-bd1ab5aef9f3 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:51.570182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-b17a108c-4c3d-4617-aa48-bd1ab5aef9f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:51.570182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:51.570182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:51.570182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:51.570182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:51.570182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:51.570182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:51.570182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:51.570182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:51.570182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:51.570182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:51.570182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:51.570182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:51.570182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:51.570182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:51.571370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:51.571370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:51.571370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:51.571370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:51.571370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:51.571370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:51.571370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:51.571370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:51.571370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:51.571370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:51.571370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:51.571370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:51.571370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:51.571370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:51.571370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:51.571370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:51.571370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:51.572478 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:51.572478 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:51.572478 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:51.572478 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:51.572723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 313/651] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:51.573541 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a108c-4c3d-4617-aa48-bd1ab5aef9f3 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:51.592273 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c16ae-0677-4a26-bee1-87799f98f2ab None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:51.645619 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4976e-e7ef-435d-bbc5-eb72fe7a48de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:51.646326 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4976e-e7ef-435d-bbc5-eb72fe7a48de None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:51.646586 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4976e-e7ef-435d-bbc5-eb72fe7a48de None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:51.646861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4976e-e7ef-435d-bbc5-eb72fe7a48de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:51.695269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a108c-4c3d-4617-aa48-bd1ab5aef9f3 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:51.736940 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4976e-e7ef-435d-bbc5-eb72fe7a48de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:51.738829 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4976e-e7ef-435d-bbc5-eb72fe7a48de None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:51.740863 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:51.741741 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d318db0-23fd-43a9-801f-3c63d4c3e401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:51.742562 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d318db0-23fd-43a9-801f-3c63d4c3e401 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:51.742844 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d318db0-23fd-43a9-801f-3c63d4c3e401 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:51.743118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d318db0-23fd-43a9-801f-3c63d4c3e401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:51.754477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:51.757922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d318db0-23fd-43a9-801f-3c63d4c3e401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:51.759441 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d318db0-23fd-43a9-801f-3c63d4c3e401 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:51.779905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-0d318db0-23fd-43a9-801f-3c63d4c3e401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:51.779905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:51.779905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:51.779905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:51.779905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:51.779905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:51.779905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:51.779905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:51.779905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:51.779905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:51.779905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:51.779905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:51.779905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:51.779905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:51.779905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:51.780872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:51.780872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:51.780872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:51.780872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:51.780872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:51.780872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:51.780872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:51.780872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:51.780872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:51.780872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:51.780872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:51.780872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:51.780872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:51.780872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:51.780872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:51.780872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:51.780872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:51.782173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:51.782173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:51.782173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:51.782173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:51.782173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 314/652] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:51.783324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d318db0-23fd-43a9-801f-3c63d4c3e401 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:51.876794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 339/653] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:51.879095 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4976e-e7ef-435d-bbc5-eb72fe7a48de None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:51.898633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d318db0-23fd-43a9-801f-3c63d4c3e401 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:51.942358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f8762-29b6-4f6b-96c1-a3e1d69d7013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:51.943822 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f8762-29b6-4f6b-96c1-a3e1d69d7013 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:51.944254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f8762-29b6-4f6b-96c1-a3e1d69d7013 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:51.944691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f8762-29b6-4f6b-96c1-a3e1d69d7013 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:51.945564 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f8762-29b6-4f6b-96c1-a3e1d69d7013 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:51.946014 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f8762-29b6-4f6b-96c1-a3e1d69d7013 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:51.948202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f8762-29b6-4f6b-96c1-a3e1d69d7013 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:51.967402 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 315/654] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:51.968580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f8762-29b6-4f6b-96c1-a3e1d69d7013 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:51.989930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4976e-e7ef-435d-bbc5-eb72fe7a48de None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:51.991474 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:52.025612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6132e-0169-41e5-b24d-2e413703b33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:52.026687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6132e-0169-41e5-b24d-2e413703b33e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:52.028113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6132e-0169-41e5-b24d-2e413703b33e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:52.028603 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6132e-0169-41e5-b24d-2e413703b33e None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:52.029533 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6132e-0169-41e5-b24d-2e413703b33e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:52.030040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6132e-0169-41e5-b24d-2e413703b33e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:52.031656 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6132e-0169-41e5-b24d-2e413703b33e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:52.035583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:52.046053 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 340/655] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:52.046991 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6132e-0169-41e5-b24d-2e413703b33e None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:52.094415 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f8762-29b6-4f6b-96c1-a3e1d69d7013 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:52.137597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a79521-cd27-45b6-a662-251a61754b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:52.138512 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a79521-cd27-45b6-a662-251a61754b1a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:52.138734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a79521-cd27-45b6-a662-251a61754b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:52.139009 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a79521-cd27-45b6-a662-251a61754b1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:52.150747 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a79521-cd27-45b6-a662-251a61754b1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:52.152376 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a79521-cd27-45b6-a662-251a61754b1a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:52.166293 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6132e-0169-41e5-b24d-2e413703b33e None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:52.170966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-35a79521-cd27-45b6-a662-251a61754b1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:52.170966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:52.170966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:52.170966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:52.170966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:52.170966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:52.170966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:52.170966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:52.170966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:52.170966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:52.170966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:52.170966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:52.170966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:52.170966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:52.170966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:52.171841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:52.171841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:52.171841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:52.171841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:52.171841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:52.171841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:52.171841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:52.171841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:52.171841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:52.171841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:52.171841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:52.171841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:52.171841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:52.171841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:52.171841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:52.171841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:52.171841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:52.173340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:52.173340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:52.173340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:52.173340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:52.173340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 316/656] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:52.173628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a79521-cd27-45b6-a662-251a61754b1a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:52.214056 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf99f23-cd9a-45c4-84a2-540f58b482dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:52.215240 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf99f23-cd9a-45c4-84a2-540f58b482dc None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:52.215761 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf99f23-cd9a-45c4-84a2-540f58b482dc None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:52.216231 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf99f23-cd9a-45c4-84a2-540f58b482dc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:52.227897 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf99f23-cd9a-45c4-84a2-540f58b482dc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:52.229966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf99f23-cd9a-45c4-84a2-540f58b482dc None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:52.250000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-1cf99f23-cd9a-45c4-84a2-540f58b482dc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 15:12:52.250000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:52.250000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:52.250000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:52.250000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:52.250000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:52.250000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:52.250000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:52.250000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:52.250000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:52.250000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:52.250000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:52.250000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:12:52.250000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:12:52.250000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:12:52.250000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:52.251019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:52.251955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:52.251955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:52.251955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:12:52.251955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:12:52.251955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:12:52.251955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:12:52.251955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:12:52.251955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:12:52.251955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 15:12:52.251955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:52.255407 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 341/657] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:52.256928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf99f23-cd9a-45c4-84a2-540f58b482dc None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:52.286634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a79521-cd27-45b6-a662-251a61754b1a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:52.289735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:52.290951 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:52.324551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e83e1-8fed-4c40-bc6b-4a7b6d9cc03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:52.325353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e83e1-8fed-4c40-bc6b-4a7b6d9cc03b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:52.325662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e83e1-8fed-4c40-bc6b-4a7b6d9cc03b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:52.325939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e83e1-8fed-4c40-bc6b-4a7b6d9cc03b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:52.334119 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e83e1-8fed-4c40-bc6b-4a7b6d9cc03b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:52.335643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e83e1-8fed-4c40-bc6b-4a7b6d9cc03b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:52.351475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-492e83e1-8fed-4c40-bc6b-4a7b6d9cc03b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 15:12:52.351475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:52.351475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:52.351475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:52.351475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:52.351475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:52.351475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:52.351475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:52.351475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:52.351475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:52.351475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:52.351475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:52.351475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:12:52.351475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:12:52.351475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:12:52.351475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:52.352442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:52.354166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:52.354166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:52.354166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:12:52.354166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:12:52.354166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:12:52.354166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:12:52.354166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:12:52.354166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:12:52.354166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 15:12:52.354166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:52.354166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 317/658] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:52.354166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e83e1-8fed-4c40-bc6b-4a7b6d9cc03b None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:52.395450 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf99f23-cd9a-45c4-84a2-540f58b482dc None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:52.437091 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad50c0e-9ed2-46f2-bbd8-336366587a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:52.437994 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad50c0e-9ed2-46f2-bbd8-336366587a3d None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:52.438446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad50c0e-9ed2-46f2-bbd8-336366587a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:52.439297 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad50c0e-9ed2-46f2-bbd8-336366587a3d None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:52.440163 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad50c0e-9ed2-46f2-bbd8-336366587a3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:52.440627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad50c0e-9ed2-46f2-bbd8-336366587a3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:52.443340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad50c0e-9ed2-46f2-bbd8-336366587a3d None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:52.458865 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 342/659] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:52.460110 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad50c0e-9ed2-46f2-bbd8-336366587a3d None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:52.466780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e83e1-8fed-4c40-bc6b-4a7b6d9cc03b None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:52.505817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864f8ca-6bc8-4dd1-a25f-b1468beffd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:52.507245 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864f8ca-6bc8-4dd1-a25f-b1468beffd28 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:52.507245 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864f8ca-6bc8-4dd1-a25f-b1468beffd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:52.507576 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864f8ca-6bc8-4dd1-a25f-b1468beffd28 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:52.508524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864f8ca-6bc8-4dd1-a25f-b1468beffd28 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:52.508936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864f8ca-6bc8-4dd1-a25f-b1468beffd28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:52.510517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864f8ca-6bc8-4dd1-a25f-b1468beffd28 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:52.529703 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 318/660] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Jul 20 15:12:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:52.535333 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864f8ca-6bc8-4dd1-a25f-b1468beffd28 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:52.561389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:52.564859 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:52.581286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad50c0e-9ed2-46f2-bbd8-336366587a3d None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:52.621110 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d5ae0-828e-40bb-8f70-2a0c92cdcc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:52.621940 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d5ae0-828e-40bb-8f70-2a0c92cdcc81 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:52.622227 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d5ae0-828e-40bb-8f70-2a0c92cdcc81 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:52.622529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d5ae0-828e-40bb-8f70-2a0c92cdcc81 None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:52.623361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d5ae0-828e-40bb-8f70-2a0c92cdcc81 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:52.623668 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d5ae0-828e-40bb-8f70-2a0c92cdcc81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:52.625359 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d5ae0-828e-40bb-8f70-2a0c92cdcc81 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:52.648428 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 343/661] 10.222.0.78 () {56 vars in 1139 bytes} [Wed Jul 20 15:12:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:52.649409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d5ae0-828e-40bb-8f70-2a0c92cdcc81 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:52.677828 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864f8ca-6bc8-4dd1-a25f-b1468beffd28 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:52.723843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765400ee-e52b-4ee9-a36d-993a17b2b651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:52.724729 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765400ee-e52b-4ee9-a36d-993a17b2b651 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:52.725079 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765400ee-e52b-4ee9-a36d-993a17b2b651 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:52.725420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765400ee-e52b-4ee9-a36d-993a17b2b651 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:52.733105 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765400ee-e52b-4ee9-a36d-993a17b2b651 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:52.734621 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765400ee-e52b-4ee9-a36d-993a17b2b651 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:52.754413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-765400ee-e52b-4ee9-a36d-993a17b2b651 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:52.754413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:52.754413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:52.754413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:52.754413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:52.754413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:52.754413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:52.754413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:52.754413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:52.754413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:52.754413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:52.754413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:52.754413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:12:52.754413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:12:52.754413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:12:52.755525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:12:52.755525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:52.755525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:52.755525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:52.755525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:52.755525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:52.755525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:52.755525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:52.755525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:52.755525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:52.755525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:52.755525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:52.755525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:52.755525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:52.755525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:52.755525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:52.755525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:52.756662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:12:52.756662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:12:52.756662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:12:52.756662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:12:52.756662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:12:52.756662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:12:52.756662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:52.756662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:52.759696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 319/662] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:52.760734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765400ee-e52b-4ee9-a36d-993a17b2b651 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:52.768036 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d5ae0-828e-40bb-8f70-2a0c92cdcc81 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:52.805283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00b39f-19b6-48d4-9b86-efbef86f43fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:52.806197 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00b39f-19b6-48d4-9b86-efbef86f43fd None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:52.806576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00b39f-19b6-48d4-9b86-efbef86f43fd None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:52.806944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00b39f-19b6-48d4-9b86-efbef86f43fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:52.822102 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:52.857611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:52.883752 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765400ee-e52b-4ee9-a36d-993a17b2b651 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:52.894547 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00b39f-19b6-48d4-9b86-efbef86f43fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:52.896322 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00b39f-19b6-48d4-9b86-efbef86f43fd None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:52.926599 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4141029-b413-49b7-ace7-c297e7a69c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:52.928039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4141029-b413-49b7-ace7-c297e7a69c56 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:52.928476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4141029-b413-49b7-ace7-c297e7a69c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:52.929031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4141029-b413-49b7-ace7-c297e7a69c56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:53.005050 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4141029-b413-49b7-ace7-c297e7a69c56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:53.006826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4141029-b413-49b7-ace7-c297e7a69c56 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:53.022085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 344/663] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:53.023232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00b39f-19b6-48d4-9b86-efbef86f43fd None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:53.075437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:53.100099 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:53.123374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 320/664] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:53.124677 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4141029-b413-49b7-ace7-c297e7a69c56 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:53.148067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00b39f-19b6-48d4-9b86-efbef86f43fd None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:53.197912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af9d98-4e97-4fa4-984b-9445393fd7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:53.198543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af9d98-4e97-4fa4-984b-9445393fd7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:53.199091 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af9d98-4e97-4fa4-984b-9445393fd7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:53.199382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af9d98-4e97-4fa4-984b-9445393fd7d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:53.200185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af9d98-4e97-4fa4-984b-9445393fd7d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:53.200523 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af9d98-4e97-4fa4-984b-9445393fd7d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:53.202868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af9d98-4e97-4fa4-984b-9445393fd7d7 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:53.224546 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 345/665] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:12:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:53.224546 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af9d98-4e97-4fa4-984b-9445393fd7d7 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:53.243870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4141029-b413-49b7-ace7-c297e7a69c56 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:53.292484 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c73a5a-435f-44ce-8004-f39e341d6f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:53.297845 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c73a5a-435f-44ce-8004-f39e341d6f4a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:53.297845 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c73a5a-435f-44ce-8004-f39e341d6f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:53.297845 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c73a5a-435f-44ce-8004-f39e341d6f4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:53.308029 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c73a5a-435f-44ce-8004-f39e341d6f4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:53.309514 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c73a5a-435f-44ce-8004-f39e341d6f4a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:53.332051 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:53.335381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-53c73a5a-435f-44ce-8004-f39e341d6f4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:53.335381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:53.335381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:53.335381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:53.335381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:53.335381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:53.335381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:53.335381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:53.335381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:53.335381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:53.335381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:53.335381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:53.335381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:53.335381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:53.335381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:53.343293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:53.343293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:53.343293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:53.343293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:53.343293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:53.343293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:53.343293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:53.343293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:53.343293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:53.343293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:53.343293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:53.343293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:53.343293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:53.343293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:53.343293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:53.343293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:53.343293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:53.344278 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:53.344278 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:53.344278 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:53.344278 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:53.344504 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af9d98-4e97-4fa4-984b-9445393fd7d7 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:53.344578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 321/666] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:53.344578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c73a5a-435f-44ce-8004-f39e341d6f4a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:53.379067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:53.392434 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe233a-f8b3-4fe2-9948-56efb278659a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:53.393246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe233a-f8b3-4fe2-9948-56efb278659a None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:53.393534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe233a-f8b3-4fe2-9948-56efb278659a None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:53.393817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe233a-f8b3-4fe2-9948-56efb278659a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:53.407189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe233a-f8b3-4fe2-9948-56efb278659a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:53.409147 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe233a-f8b3-4fe2-9948-56efb278659a None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:53.432534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-8efe233a-f8b3-4fe2-9948-56efb278659a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:53.432534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:53.432534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:53.432534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:53.432534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:53.432534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:53.432534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:53.432534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:53.432534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:53.432534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:53.432534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:53.432534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:53.432534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:53.432534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:53.432534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:53.433561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:53.433561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:53.433561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:53.433561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:53.433561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:53.433561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:53.433561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:53.433561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:53.433561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:53.433561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:53.433561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:53.433561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:53.433561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:53.433561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:53.433561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:53.433561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:53.433561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:53.434579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:53.434579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:53.434579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:53.434579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:53.434579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 346/667] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:53.435322 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe233a-f8b3-4fe2-9948-56efb278659a None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:53.457398 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c73a5a-435f-44ce-8004-f39e341d6f4a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:53.494919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478438c8-1ecd-4594-a476-0f5c201ee87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:53.495582 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478438c8-1ecd-4594-a476-0f5c201ee87a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:53.495880 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478438c8-1ecd-4594-a476-0f5c201ee87a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:53.496155 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478438c8-1ecd-4594-a476-0f5c201ee87a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:53.496995 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478438c8-1ecd-4594-a476-0f5c201ee87a None admin] RBAC: Adding query filter params (role.id=d820b1d3fa4144aca575c04645c099bc, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=b6c13596c2df438294bdac7c52ae8cb1) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:53.497312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478438c8-1ecd-4594-a476-0f5c201ee87a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:53.498781 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478438c8-1ecd-4594-a476-0f5c201ee87a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:53.514286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 322/668] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:12:53 2022] GET /identity/v3/role_assignments?user.id=b6c13596c2df438294bdac7c52ae8cb1&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=d820b1d3fa4144aca575c04645c099bc => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:53.514286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478438c8-1ecd-4594-a476-0f5c201ee87a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:53.558923 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe233a-f8b3-4fe2-9948-56efb278659a None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:53.602023 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b62d0-fae3-4a64-82c2-dee96d2e5fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:53.602747 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b62d0-fae3-4a64-82c2-dee96d2e5fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:53.603068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b62d0-fae3-4a64-82c2-dee96d2e5fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:53.603354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b62d0-fae3-4a64-82c2-dee96d2e5fa4 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:53.604098 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b62d0-fae3-4a64-82c2-dee96d2e5fa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:53.604410 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b62d0-fae3-4a64-82c2-dee96d2e5fa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:53.605823 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b62d0-fae3-4a64-82c2-dee96d2e5fa4 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:53.619090 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 347/669] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:53.620503 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eddc74-c879-40b9-bf8a-c93730ca53f9 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:53.620747 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eddc74-c879-40b9-bf8a-c93730ca53f9 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:53.621029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eddc74-c879-40b9-bf8a-c93730ca53f9 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:53.626604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 348/670] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:53.628339 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11944e-6c1b-43ba-aabf-b7cd898e6e19 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:53.628636 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11944e-6c1b-43ba-aabf-b7cd898e6e19 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:53.628925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11944e-6c1b-43ba-aabf-b7cd898e6e19 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:53.634886 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 349/671] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:53.634886 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11944e-6c1b-43ba-aabf-b7cd898e6e19 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:53.647423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:53.648533 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:53.651179 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478438c8-1ecd-4594-a476-0f5c201ee87a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:53.687671 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c020ec-553d-4288-9814-8eb94f22bb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:53.688350 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c020ec-553d-4288-9814-8eb94f22bb2b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:53.688633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c020ec-553d-4288-9814-8eb94f22bb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:53.688906 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c020ec-553d-4288-9814-8eb94f22bb2b None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:53.689632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c020ec-553d-4288-9814-8eb94f22bb2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:53.689936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c020ec-553d-4288-9814-8eb94f22bb2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:53.691364 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c020ec-553d-4288-9814-8eb94f22bb2b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:53.703662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 323/672] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:53.705178 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba6489-113a-40bb-90b6-167aa25c76da None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:53.705479 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba6489-113a-40bb-90b6-167aa25c76da None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:53.705764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba6489-113a-40bb-90b6-167aa25c76da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:53.724766 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-afba6489-113a-40bb-90b6-167aa25c76da None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:53.725752 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-afba6489-113a-40bb-90b6-167aa25c76da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:53.754049 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11944e-6c1b-43ba-aabf-b7cd898e6e19 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:53.794421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181efc34-0d49-4f96-af4f-94221a1833ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:53.795383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181efc34-0d49-4f96-af4f-94221a1833ba None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:53.795666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181efc34-0d49-4f96-af4f-94221a1833ba None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:53.796149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181efc34-0d49-4f96-af4f-94221a1833ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:53.810532 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 324/673] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:53.810692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181efc34-0d49-4f96-af4f-94221a1833ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:53.810772 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46deed7b-2722-4573-9c83-88a916ac8353 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:53.810772 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46deed7b-2722-4573-9c83-88a916ac8353 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:53.811001 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181efc34-0d49-4f96-af4f-94221a1833ba None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:53.811076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46deed7b-2722-4573-9c83-88a916ac8353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:53.825419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-46deed7b-2722-4573-9c83-88a916ac8353 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:53.826018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-181efc34-0d49-4f96-af4f-94221a1833ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:53.826018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:53.826018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:53.826018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:53.826018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:53.826018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:53.826018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:53.826018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:53.826018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:53.826018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:53.826018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:53.826018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:53.826018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:12:53.826018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:12:53.826018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:12:53.826937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:12:53.826937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:53.826937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:53.826937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:53.826937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:53.826937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:53.826937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:53.826937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:53.826937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:53.826937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:53.826937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:53.826937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:53.826937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:53.826937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:53.826937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:53.826937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:53.826937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:53.827844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:12:53.827844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:12:53.827844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:12:53.827844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:12:53.827844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:12:53.827844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:12:53.827844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:53.827844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:53.828286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-46deed7b-2722-4573-9c83-88a916ac8353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:53.832481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 350/674] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:53.833350 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181efc34-0d49-4f96-af4f-94221a1833ba None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:53.905652 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 325/675] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:53.907449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46deed7b-2722-4573-9c83-88a916ac8353 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:53.910987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:53.935215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:53.941922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181efc34-0d49-4f96-af4f-94221a1833ba None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:53.978523 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d4526-fddd-4f74-868f-bb171e7aba60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:53.979450 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d4526-fddd-4f74-868f-bb171e7aba60 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:53.979728 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d4526-fddd-4f74-868f-bb171e7aba60 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:53.979997 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d4526-fddd-4f74-868f-bb171e7aba60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:53.986985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d4526-fddd-4f74-868f-bb171e7aba60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:53.988512 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d4526-fddd-4f74-868f-bb171e7aba60 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:54.003819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-c09d4526-fddd-4f74-868f-bb171e7aba60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:54.003819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:54.003819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:54.003819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:54.003819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:54.003819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:54.003819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:54.003819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:54.003819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:54.003819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:54.003819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:54.003819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:54.003819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:12:54.003819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:12:54.003819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:12:54.004933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:12:54.004933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:54.004933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:54.004933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:54.004933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:54.004933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:54.004933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:54.004933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:54.004933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:54.004933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:54.004933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:54.004933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:54.004933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:54.004933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:54.004933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:54.004933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:54.004933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:54.005856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:12:54.005856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:12:54.005856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:12:54.005856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:12:54.005856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:12:54.005856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:12:54.005856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:54.005856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:54.005856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 351/676] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:54.005856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d4526-fddd-4f74-868f-bb171e7aba60 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:54.038860 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46deed7b-2722-4573-9c83-88a916ac8353 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:54.077391 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7d97d-08a9-4145-923d-252bdbe03954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:54.078081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7d97d-08a9-4145-923d-252bdbe03954 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:54.078402 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7d97d-08a9-4145-923d-252bdbe03954 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:54.078681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7d97d-08a9-4145-923d-252bdbe03954 None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:54.079482 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b7d97d-08a9-4145-923d-252bdbe03954 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:54.079799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b7d97d-08a9-4145-923d-252bdbe03954 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:54.081225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b7d97d-08a9-4145-923d-252bdbe03954 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:54.093693 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 326/677] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:12:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:54.094901 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7d97d-08a9-4145-923d-252bdbe03954 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:54.121209 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d4526-fddd-4f74-868f-bb171e7aba60 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:54.151960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:54.156741 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b39a8d-ab45-4c32-bb3d-a05dd55cf9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:54.157540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b39a8d-ab45-4c32-bb3d-a05dd55cf9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:54.157821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b39a8d-ab45-4c32-bb3d-a05dd55cf9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:54.158091 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b39a8d-ab45-4c32-bb3d-a05dd55cf9c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:54.168407 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b39a8d-ab45-4c32-bb3d-a05dd55cf9c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:54.176018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-81b39a8d-ab45-4c32-bb3d-a05dd55cf9c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:54.176018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:54.176018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:54.176018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:54.176018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:54.176018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:54.176018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:54.176018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:54.176018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:54.176018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:54.176018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:54.176018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:54.176018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:54.176018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:54.176018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:54.176018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:54.176985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:54.176985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:54.176985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:54.176985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:54.176985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:54.176985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:54.176985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:54.176985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:54.176985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:54.176985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:54.176985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:54.176985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:54.176985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:54.176985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:54.176985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:54.176985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:54.177840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:54.177840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:54.177840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:54.177840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:54.177840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:54.177840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:54.177840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:54.177840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:54.177840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:54.177840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:54.177840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:54.177840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:54.177840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 352/678] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:54.178596 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b39a8d-ab45-4c32-bb3d-a05dd55cf9c3 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:54.187110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:54.209524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7d97d-08a9-4145-923d-252bdbe03954 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:54.247077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3cf34-b929-42bc-8d97-4d123a09489c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:54.247791 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3cf34-b929-42bc-8d97-4d123a09489c None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:54.248073 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3cf34-b929-42bc-8d97-4d123a09489c None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:54.248346 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3cf34-b929-42bc-8d97-4d123a09489c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:54.260218 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3cf34-b929-42bc-8d97-4d123a09489c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:54.267761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-73b3cf34-b929-42bc-8d97-4d123a09489c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:12:54.267761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:54.267761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:54.267761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:54.267761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:54.267761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:54.267761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:54.267761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:54.267761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:54.267761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:54.267761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:54.267761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:54.267761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:54.267761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:54.267761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:54.267761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:54.268735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:54.268735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:54.268735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:54.268735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:54.268735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:54.268735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:54.268735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:54.268735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:54.268735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:54.268735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:54.268735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:54.268735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:54.268735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:54.268735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:54.268735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:54.268735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:54.269639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:54.269639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:54.269639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:54.269639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:54.269639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:54.269639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:54.269639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:54.269639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:54.269639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:54.269639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:54.269639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:12:54.269639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:54.269639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 327/679] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:12:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:54.270985 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3cf34-b929-42bc-8d97-4d123a09489c None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:54.292941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b39a8d-ab45-4c32-bb3d-a05dd55cf9c3 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:54.336446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7213390-a2ee-464c-8d81-20d36ffa750e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:54.337113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7213390-a2ee-464c-8d81-20d36ffa750e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:54.337394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7213390-a2ee-464c-8d81-20d36ffa750e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:54.337670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7213390-a2ee-464c-8d81-20d36ffa750e None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:54.338491 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7213390-a2ee-464c-8d81-20d36ffa750e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:54.338796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7213390-a2ee-464c-8d81-20d36ffa750e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:54.340238 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7213390-a2ee-464c-8d81-20d36ffa750e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:54.358042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 353/680] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:12:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:54.358042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7213390-a2ee-464c-8d81-20d36ffa750e None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:54.377440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3cf34-b929-42bc-8d97-4d123a09489c None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:54.412658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:54.412910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df1925-897d-43f8-87f3-c84ad8872e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:54.413508 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df1925-897d-43f8-87f3-c84ad8872e4a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:54.413792 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df1925-897d-43f8-87f3-c84ad8872e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:54.414094 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df1925-897d-43f8-87f3-c84ad8872e4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:54.415081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df1925-897d-43f8-87f3-c84ad8872e4a None admin] RBAC: Adding query filter params (role.id=d820b1d3fa4144aca575c04645c099bc, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=b51e9bdab4164d87b7b0b11c505c2c0a) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:54.415310 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df1925-897d-43f8-87f3-c84ad8872e4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:54.416728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df1925-897d-43f8-87f3-c84ad8872e4a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:54.422826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 328/681] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:12:54 2022] GET /identity/v3/role_assignments?user.id=b51e9bdab4164d87b7b0b11c505c2c0a&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=d820b1d3fa4144aca575c04645c099bc => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:54.423868 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df1925-897d-43f8-87f3-c84ad8872e4a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:54.427273 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:54.489468 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7213390-a2ee-464c-8d81-20d36ffa750e None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:54.543814 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064ae4d-0c29-4527-8228-685aa1d77d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:54.544511 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064ae4d-0c29-4527-8228-685aa1d77d11 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:54.544800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064ae4d-0c29-4527-8228-685aa1d77d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:54.545116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064ae4d-0c29-4527-8228-685aa1d77d11 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:54.546539 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064ae4d-0c29-4527-8228-685aa1d77d11 None admin] RBAC: Adding query filter params (name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:54.546963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064ae4d-0c29-4527-8228-685aa1d77d11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:54.548366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064ae4d-0c29-4527-8228-685aa1d77d11 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:54.549189 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df1925-897d-43f8-87f3-c84ad8872e4a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:54.564703 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 354/682] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:12:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:54.566608 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064ae4d-0c29-4527-8228-685aa1d77d11 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:54.605548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536bc73-14e6-42c5-9944-126db12f991d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:54.605548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536bc73-14e6-42c5-9944-126db12f991d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:54.605840 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536bc73-14e6-42c5-9944-126db12f991d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:54.606086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536bc73-14e6-42c5-9944-126db12f991d None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:54.606863 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536bc73-14e6-42c5-9944-126db12f991d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:54.607170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536bc73-14e6-42c5-9944-126db12f991d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:54.608519 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536bc73-14e6-42c5-9944-126db12f991d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:54.619999 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 329/683] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:12:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:54.620804 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536bc73-14e6-42c5-9944-126db12f991d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:54.692390 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:54.705740 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064ae4d-0c29-4527-8228-685aa1d77d11 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:54.724778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:54.737207 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536bc73-14e6-42c5-9944-126db12f991d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:54.755199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b282e1-0725-40e0-9f6d-101fd5c9f5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:54.756831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b282e1-0725-40e0-9f6d-101fd5c9f5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:54.757399 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b282e1-0725-40e0-9f6d-101fd5c9f5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:54.758150 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b282e1-0725-40e0-9f6d-101fd5c9f5c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:54.759407 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b282e1-0725-40e0-9f6d-101fd5c9f5c9 None admin] RBAC: Adding query filter params (role.id=d820b1d3fa4144aca575c04645c099bc, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=b57530f18d9c4d328f8014db736d8de9) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:54.760240 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b282e1-0725-40e0-9f6d-101fd5c9f5c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:54.762067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b282e1-0725-40e0-9f6d-101fd5c9f5c9 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:54.779199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 355/684] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:12:54 2022] GET /identity/v3/role_assignments?user.id=b57530f18d9c4d328f8014db736d8de9&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=d820b1d3fa4144aca575c04645c099bc => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:54.779199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b282e1-0725-40e0-9f6d-101fd5c9f5c9 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:54.795174 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a9571-814c-4105-ac26-6e568207fa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:54.795900 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a9571-814c-4105-ac26-6e568207fa66 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:54.796186 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a9571-814c-4105-ac26-6e568207fa66 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:54.796463 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a9571-814c-4105-ac26-6e568207fa66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:54.825436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a9571-814c-4105-ac26-6e568207fa66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:54.826959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a9571-814c-4105-ac26-6e568207fa66 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:54.865586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-940a9571-814c-4105-ac26-6e568207fa66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:54.865586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:54.865586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:54.865586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:54.865586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:54.865586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:54.865586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:54.865586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:54.865586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:54.865586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:54.865586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:54.865586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:54.865586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:54.865586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:54.865586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:54.866633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:54.866633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:54.866633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:54.866633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:54.866633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:54.866633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:54.866633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:54.866633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:54.866633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:54.866633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:54.866633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:54.866633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:54.866633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:54.866633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:54.866633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:54.866633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:54.866633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:54.867566 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:54.867566 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:54.867566 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:54.867566 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:54.875616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 330/685] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:54.881383 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a9571-814c-4105-ac26-6e568207fa66 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:54.991286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a9571-814c-4105-ac26-6e568207fa66 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:55.007549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:55.015674 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b282e1-0725-40e0-9f6d-101fd5c9f5c9 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:55.033848 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863c264-8b24-4fa1-be4c-b47d2945a257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:55.034630 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863c264-8b24-4fa1-be4c-b47d2945a257 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:55.035410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863c264-8b24-4fa1-be4c-b47d2945a257 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:55.035687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863c264-8b24-4fa1-be4c-b47d2945a257 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:55.036421 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863c264-8b24-4fa1-be4c-b47d2945a257 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:55.036731 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863c264-8b24-4fa1-be4c-b47d2945a257 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:55.038126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863c264-8b24-4fa1-be4c-b47d2945a257 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:55.052653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 331/686] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:55.058246 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863c264-8b24-4fa1-be4c-b47d2945a257 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:55.059795 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eff5ad-5f86-44ae-a95b-50f85c883161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:55.061149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eff5ad-5f86-44ae-a95b-50f85c883161 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:55.061644 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eff5ad-5f86-44ae-a95b-50f85c883161 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:55.062165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eff5ad-5f86-44ae-a95b-50f85c883161 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:55.077772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eff5ad-5f86-44ae-a95b-50f85c883161 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:55.080146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eff5ad-5f86-44ae-a95b-50f85c883161 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:55.104388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-29eff5ad-5f86-44ae-a95b-50f85c883161 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:55.104388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:55.104388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:55.104388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:55.104388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:55.104388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:55.104388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:55.104388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:55.104388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:55.104388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:55.104388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:55.104388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:55.104388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:55.104388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:55.104388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:55.105301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:55.105301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:55.105301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:55.105301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:55.105301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:55.105301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:55.105301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:55.105301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:55.105301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:55.105301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:55.105301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:55.105301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:55.105301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:55.105301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:55.105301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:55.105301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:55.105301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:55.106253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:55.106253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:55.106253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:55.106253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:55.106253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 356/687] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:55.111522 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eff5ad-5f86-44ae-a95b-50f85c883161 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:55.142033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:55.189367 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863c264-8b24-4fa1-be4c-b47d2945a257 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:55.239287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b133b-bcbc-4fa6-9d1b-df5f3949e6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:55.240555 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b133b-bcbc-4fa6-9d1b-df5f3949e6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:55.241075 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b133b-bcbc-4fa6-9d1b-df5f3949e6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:55.241597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b133b-bcbc-4fa6-9d1b-df5f3949e6a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:55.247239 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eff5ad-5f86-44ae-a95b-50f85c883161 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:55.250745 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b133b-bcbc-4fa6-9d1b-df5f3949e6a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:55.252464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b133b-bcbc-4fa6-9d1b-df5f3949e6a6 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:55.269516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-fc7b133b-bcbc-4fa6-9d1b-df5f3949e6a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:12:55.269516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:55.269516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:55.269516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:55.269516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:55.269516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:55.269516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:55.269516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:55.269516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:55.269516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:55.269516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:55.269516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:55.269516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:12:55.269516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:12:55.269516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:12:55.269516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:55.270574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:55.271575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:55.271575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:55.271575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:12:55.271575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:12:55.271575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:12:55.271575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:12:55.271575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:12:55.271575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:12:55.271575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:12:55.271575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:55.274329 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 332/688] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:12:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:55.282592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b133b-bcbc-4fa6-9d1b-df5f3949e6a6 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:55.318827 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:55.380412 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18ce85-7512-4f23-92a3-efcc8b9a1cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:55.380896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18ce85-7512-4f23-92a3-efcc8b9a1cfa None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:55.381177 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18ce85-7512-4f23-92a3-efcc8b9a1cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:55.381453 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18ce85-7512-4f23-92a3-efcc8b9a1cfa None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:55.382243 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18ce85-7512-4f23-92a3-efcc8b9a1cfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:55.382493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18ce85-7512-4f23-92a3-efcc8b9a1cfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:55.383877 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18ce85-7512-4f23-92a3-efcc8b9a1cfa None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:55.394907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b133b-bcbc-4fa6-9d1b-df5f3949e6a6 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:55.416856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 357/689] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:55.423637 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18ce85-7512-4f23-92a3-efcc8b9a1cfa None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:55.439584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f14b88-d11c-4728-aff4-76d96186791f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:55.441568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f14b88-d11c-4728-aff4-76d96186791f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:55.442012 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f14b88-d11c-4728-aff4-76d96186791f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:55.442481 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f14b88-d11c-4728-aff4-76d96186791f None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:55.443533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f14b88-d11c-4728-aff4-76d96186791f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:55.444011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f14b88-d11c-4728-aff4-76d96186791f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:55.445636 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f14b88-d11c-4728-aff4-76d96186791f None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:55.465762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 333/690] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Jul 20 15:12:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:55.472011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f14b88-d11c-4728-aff4-76d96186791f None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:55.578206 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f14b88-d11c-4728-aff4-76d96186791f None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:55.602512 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:55.616616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713df9d-1828-4081-bdee-4e1c75a6fe60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:55.617497 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713df9d-1828-4081-bdee-4e1c75a6fe60 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:55.617953 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713df9d-1828-4081-bdee-4e1c75a6fe60 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:55.618412 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713df9d-1828-4081-bdee-4e1c75a6fe60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:55.694213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713df9d-1828-4081-bdee-4e1c75a6fe60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:55.696059 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713df9d-1828-4081-bdee-4e1c75a6fe60 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:55.758312 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:55.773849 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18ce85-7512-4f23-92a3-efcc8b9a1cfa None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:55.807581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:55.810893 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6799b5-d4c2-435c-ae3e-904b1266c14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:55.811860 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6799b5-d4c2-435c-ae3e-904b1266c14c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:55.812315 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6799b5-d4c2-435c-ae3e-904b1266c14c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:55.812778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6799b5-d4c2-435c-ae3e-904b1266c14c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:55.818366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 334/691] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 15:12:55.818366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e85c6-c415-462f-9a41-9d74d36a1f4d None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:55.818366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e85c6-c415-462f-9a41-9d74d36a1f4d None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:55.818366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e85c6-c415-462f-9a41-9d74d36a1f4d None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:55.823756 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 335/692] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:55.824542 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e85c6-c415-462f-9a41-9d74d36a1f4d None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:55.825694 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6799b5-d4c2-435c-ae3e-904b1266c14c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:55.827206 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6799b5-d4c2-435c-ae3e-904b1266c14c None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:55.846652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-6a6799b5-d4c2-435c-ae3e-904b1266c14c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:12:55.846652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:55.846652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:55.846652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:55.846652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:55.846652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:55.846652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:55.846652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:55.846652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:55.846652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:55.846652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:55.846652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:55.846652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:12:55.846652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:12:55.846652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:12:55.846652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:55.848354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:55.849382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:55.849382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:55.849382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:12:55.849382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:12:55.849382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:12:55.849382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:12:55.849382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:12:55.849382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:12:55.849382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:12:55.849382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:55.853502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 358/693] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Jul 20 15:12:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:12:55.855105 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1207e8c-40c1-4317-9141-7450ecf6a7ad None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:55.855401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1207e8c-40c1-4317-9141-7450ecf6a7ad None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:55.855695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1207e8c-40c1-4317-9141-7450ecf6a7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:55.878496 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-f1207e8c-40c1-4317-9141-7450ecf6a7ad None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:55.880334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f1207e8c-40c1-4317-9141-7450ecf6a7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:55.945107 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e85c6-c415-462f-9a41-9d74d36a1f4d None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:55.955106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 359/694] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:55.956115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1207e8c-40c1-4317-9141-7450ecf6a7ad None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:56.012507 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd590f30-59fe-47d0-a708-35e3a2f58c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:56.013439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd590f30-59fe-47d0-a708-35e3a2f58c0b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:56.013729 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd590f30-59fe-47d0-a708-35e3a2f58c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:56.014011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd590f30-59fe-47d0-a708-35e3a2f58c0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:56.026143 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd590f30-59fe-47d0-a708-35e3a2f58c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:56.027690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd590f30-59fe-47d0-a708-35e3a2f58c0b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:56.046021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:56.047944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-dd590f30-59fe-47d0-a708-35e3a2f58c0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:56.047944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:56.047944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:56.047944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:56.047944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:56.047944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:56.047944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:56.047944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:56.047944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:56.047944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:56.047944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:56.047944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:56.047944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:56.047944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:56.047944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:56.048856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:56.048856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:56.048856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:56.048856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:56.048856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:56.048856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:56.048856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:56.048856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:56.048856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:56.048856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:56.048856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:56.048856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:56.048856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:56.048856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:56.048856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:56.048856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:56.048856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:56.049777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:56.049777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:56.049777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:56.049777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:56.049777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 336/695] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:56.050515 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd590f30-59fe-47d0-a708-35e3a2f58c0b None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:56.066287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1207e8c-40c1-4317-9141-7450ecf6a7ad None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:56.103950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50101a2-6b9b-4c06-a006-8dd6487c8a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:56.104649 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50101a2-6b9b-4c06-a006-8dd6487c8a7d None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:56.104930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50101a2-6b9b-4c06-a006-8dd6487c8a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:56.105207 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50101a2-6b9b-4c06-a006-8dd6487c8a7d None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:56.106196 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50101a2-6b9b-4c06-a006-8dd6487c8a7d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:56.106562 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50101a2-6b9b-4c06-a006-8dd6487c8a7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:56.108052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50101a2-6b9b-4c06-a006-8dd6487c8a7d None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:56.115918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:56.129460 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 360/696] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Jul 20 15:12:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:56.130562 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50101a2-6b9b-4c06-a006-8dd6487c8a7d None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:56.185232 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd590f30-59fe-47d0-a708-35e3a2f58c0b None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:56.231549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25bab21-91af-4523-98ac-3b9e1e15a5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:56.232996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bab21-91af-4523-98ac-3b9e1e15a5bf None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:56.233690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bab21-91af-4523-98ac-3b9e1e15a5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:56.234112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bab21-91af-4523-98ac-3b9e1e15a5bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:56.248835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25bab21-91af-4523-98ac-3b9e1e15a5bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:56.251091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25bab21-91af-4523-98ac-3b9e1e15a5bf None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:56.256111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50101a2-6b9b-4c06-a006-8dd6487c8a7d None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:56.274354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-b25bab21-91af-4523-98ac-3b9e1e15a5bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:56.274354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:56.274354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:56.274354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:56.274354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:56.274354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:56.274354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:56.274354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:56.274354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:56.274354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:56.274354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:56.274354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:56.274354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:56.274354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:56.274354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:56.275485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:56.275485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:56.275485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:56.275485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:56.275485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:56.275485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:56.275485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:56.275485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:56.275485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:56.275485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:56.275485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:56.275485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:56.275485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:56.275485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:56.275485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:56.275485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:56.275485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:56.276482 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:56.276482 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:56.276482 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:56.276482 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:56.279073 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 337/697] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:56.280111 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25bab21-91af-4523-98ac-3b9e1e15a5bf None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:56.300135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1fd05-abe3-4210-98ac-5b1860a5ad0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:56.301368 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1fd05-abe3-4210-98ac-5b1860a5ad0d None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:56.301719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1fd05-abe3-4210-98ac-5b1860a5ad0d None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:56.301933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1fd05-abe3-4210-98ac-5b1860a5ad0d None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:56.302715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1fd05-abe3-4210-98ac-5b1860a5ad0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:56.303026 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1fd05-abe3-4210-98ac-5b1860a5ad0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:56.304620 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1fd05-abe3-4210-98ac-5b1860a5ad0d None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:56.311563 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:56.318067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 361/698] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:56.319000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1fd05-abe3-4210-98ac-5b1860a5ad0d None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:56.394730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:56.400199 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25bab21-91af-4523-98ac-3b9e1e15a5bf None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:56.436436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fe29c-50ed-4f69-89ad-675fa91ac2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:56.437095 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fe29c-50ed-4f69-89ad-675fa91ac2ad None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:56.437384 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fe29c-50ed-4f69-89ad-675fa91ac2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:56.437663 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fe29c-50ed-4f69-89ad-675fa91ac2ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:56.461712 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1fd05-abe3-4210-98ac-5b1860a5ad0d None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:56.501566 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a3934-0315-4c79-b347-bcfa74c30616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:56.502287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a3934-0315-4c79-b347-bcfa74c30616 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:56.502889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a3934-0315-4c79-b347-bcfa74c30616 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:56.503464 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a3934-0315-4c79-b347-bcfa74c30616 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:56.504115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a3934-0315-4c79-b347-bcfa74c30616 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:56.504448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a3934-0315-4c79-b347-bcfa74c30616 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:56.505798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a3934-0315-4c79-b347-bcfa74c30616 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:56.518734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fe29c-50ed-4f69-89ad-675fa91ac2ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:56.521098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fe29c-50ed-4f69-89ad-675fa91ac2ad None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:56.523719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 362/699] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:56.525216 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a3934-0315-4c79-b347-bcfa74c30616 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:56.587562 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:56.633413 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a3934-0315-4c79-b347-bcfa74c30616 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:56.654633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:56.662196 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 338/700] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:12:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:56.663627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fe29c-50ed-4f69-89ad-675fa91ac2ad None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:56.678462 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791ea0b-c6f0-47f1-99f1-3e7e1d336487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:56.679271 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791ea0b-c6f0-47f1-99f1-3e7e1d336487 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:56.679561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791ea0b-c6f0-47f1-99f1-3e7e1d336487 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:56.679844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791ea0b-c6f0-47f1-99f1-3e7e1d336487 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:56.688100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791ea0b-c6f0-47f1-99f1-3e7e1d336487 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:56.689869 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791ea0b-c6f0-47f1-99f1-3e7e1d336487 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:56.705930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-d791ea0b-c6f0-47f1-99f1-3e7e1d336487 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:56.705930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:56.705930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:56.705930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:56.705930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:56.705930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:56.705930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:56.705930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:56.705930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:56.705930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:56.705930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:56.705930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:56.705930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:12:56.705930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:12:56.705930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:12:56.707499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:12:56.707499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:56.707499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:56.707499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:56.707499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:56.707499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:56.707499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:56.707499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:56.707499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:56.707499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:56.707499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:56.707499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:56.707499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:56.707499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:56.707499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:56.707499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:56.707499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:56.708798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:12:56.708798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:12:56.708798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:12:56.708798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:12:56.708798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:12:56.708798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:12:56.708798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:56.708798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:56.708798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 363/701] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:56.709455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791ea0b-c6f0-47f1-99f1-3e7e1d336487 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:56.790524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fe29c-50ed-4f69-89ad-675fa91ac2ad None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:56.824468 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791ea0b-c6f0-47f1-99f1-3e7e1d336487 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:56.834599 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ba95e-f7e0-4c3e-bc85-7594c3b9c03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:56.834939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ba95e-f7e0-4c3e-bc85-7594c3b9c03a None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:56.835278 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ba95e-f7e0-4c3e-bc85-7594c3b9c03a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:56.835560 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ba95e-f7e0-4c3e-bc85-7594c3b9c03a None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:56.836328 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ba95e-f7e0-4c3e-bc85-7594c3b9c03a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:56.837778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ba95e-f7e0-4c3e-bc85-7594c3b9c03a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:56.845696 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:56.864342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e39d1-1a0b-446e-8301-c21cb8cc7047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:56.865243 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e39d1-1a0b-446e-8301-c21cb8cc7047 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:56.865521 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e39d1-1a0b-446e-8301-c21cb8cc7047 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:56.865792 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e39d1-1a0b-446e-8301-c21cb8cc7047 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:56.875635 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e39d1-1a0b-446e-8301-c21cb8cc7047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:56.877244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e39d1-1a0b-446e-8301-c21cb8cc7047 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:56.895630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-3e7e39d1-1a0b-446e-8301-c21cb8cc7047 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:56.895630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:56.895630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:56.895630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:56.895630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:56.895630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:56.895630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:56.895630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:56.895630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:56.895630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:56.895630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:56.895630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:56.895630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:56.895630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:56.895630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:56.896799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:56.896799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:56.896799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:56.896799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:56.896799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:56.896799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:56.896799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:56.896799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:56.896799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:56.896799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:56.896799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:56.896799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:56.896799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:56.896799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:56.896799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:56.896799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:56.896799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:56.897873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:56.897873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:56.897873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:56.897873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:56.897873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 364/702] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:56.898210 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e39d1-1a0b-446e-8301-c21cb8cc7047 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:56.936480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 339/703] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:12:56 2022] POST /identity/v3/users => generated 254 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:12:56.937861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d34d6-7487-420b-b90f-196629fb68f6 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:56.938276 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d34d6-7487-420b-b90f-196629fb68f6 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:56.938820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d34d6-7487-420b-b90f-196629fb68f6 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:56.944784 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 340/704] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:56.944784 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d34d6-7487-420b-b90f-196629fb68f6 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:57.015123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e39d1-1a0b-446e-8301-c21cb8cc7047 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:57.029651 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:57.056069 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037eb87-2083-4277-88c6-bafe961d9994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:57.056750 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037eb87-2083-4277-88c6-bafe961d9994 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:57.057034 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037eb87-2083-4277-88c6-bafe961d9994 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:57.057308 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037eb87-2083-4277-88c6-bafe961d9994 None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:57.058291 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037eb87-2083-4277-88c6-bafe961d9994 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:57.058486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037eb87-2083-4277-88c6-bafe961d9994 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:57.059927 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037eb87-2083-4277-88c6-bafe961d9994 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:57.073054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 365/705] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:12:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:57.076108 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d421b-37aa-4b77-9dd5-9b9d1b42c69b None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:57.077079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d421b-37aa-4b77-9dd5-9b9d1b42c69b None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:57.077951 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d421b-37aa-4b77-9dd5-9b9d1b42c69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:57.086977 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d34d6-7487-420b-b90f-196629fb68f6 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:57.096991 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-db8d421b-37aa-4b77-9dd5-9b9d1b42c69b None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:57.098209 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-db8d421b-37aa-4b77-9dd5-9b9d1b42c69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:57.125506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:57.132317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f442f-ef30-4d47-8e4e-674053121526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:57.133065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f442f-ef30-4d47-8e4e-674053121526 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:57.133419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f442f-ef30-4d47-8e4e-674053121526 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:57.133757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f442f-ef30-4d47-8e4e-674053121526 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:57.134624 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f442f-ef30-4d47-8e4e-674053121526 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:57.134995 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f442f-ef30-4d47-8e4e-674053121526 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:57.136459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f442f-ef30-4d47-8e4e-674053121526 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:57.149459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 341/706] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:57.150862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380f18e-0017-485e-902a-65af8513b280 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:57.152088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380f18e-0017-485e-902a-65af8513b280 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:57.152088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380f18e-0017-485e-902a-65af8513b280 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:57.153014 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 342/707] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:57.154287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380f18e-0017-485e-902a-65af8513b280 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:57.187803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 366/708] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:57.189403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cc18e-e664-46ac-a617-d8cf409a9765 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:57.189695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cc18e-e664-46ac-a617-d8cf409a9765 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:57.189972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cc18e-e664-46ac-a617-d8cf409a9765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:57.207790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-884cc18e-e664-46ac-a617-d8cf409a9765 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:57.209081 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-884cc18e-e664-46ac-a617-d8cf409a9765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:57.272972 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380f18e-0017-485e-902a-65af8513b280 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:57.284548 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 367/709] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:57.285987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cc18e-e664-46ac-a617-d8cf409a9765 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:57.293196 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:57.317457 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4dd68-3416-40d8-804d-bcfa4b6472e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:57.318116 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4dd68-3416-40d8-804d-bcfa4b6472e1 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:57.318426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4dd68-3416-40d8-804d-bcfa4b6472e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:57.318711 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4dd68-3416-40d8-804d-bcfa4b6472e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:57.319581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4dd68-3416-40d8-804d-bcfa4b6472e1 None admin] RBAC: Adding query filter params (role.id=d820b1d3fa4144aca575c04645c099bc, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=ed0d719c0eb04a3b8a228c6ce70fdf98) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:57.319851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4dd68-3416-40d8-804d-bcfa4b6472e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:57.321414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4dd68-3416-40d8-804d-bcfa4b6472e1 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:57.327530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 343/710] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:12:57 2022] GET /identity/v3/role_assignments?user.id=ed0d719c0eb04a3b8a228c6ce70fdf98&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=d820b1d3fa4144aca575c04645c099bc => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:57.335747 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4dd68-3416-40d8-804d-bcfa4b6472e1 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:57.392643 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:57.404365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cc18e-e664-46ac-a617-d8cf409a9765 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:57.446201 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c8255-3d87-410c-ae8b-44b3345e96fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:57.447059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c8255-3d87-410c-ae8b-44b3345e96fc None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:57.447325 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c8255-3d87-410c-ae8b-44b3345e96fc None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:57.447602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c8255-3d87-410c-ae8b-44b3345e96fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:57.457053 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c8255-3d87-410c-ae8b-44b3345e96fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:57.459348 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c8255-3d87-410c-ae8b-44b3345e96fc None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:57.464307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4dd68-3416-40d8-804d-bcfa4b6472e1 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:57.476641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-644c8255-3d87-410c-ae8b-44b3345e96fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:57.476641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:57.476641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:57.476641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:57.476641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:57.476641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:57.476641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:57.476641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:57.476641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:57.476641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:57.476641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:57.476641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:57.476641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:12:57.476641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:12:57.476641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:12:57.477653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:12:57.477653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:57.477653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:57.477653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:57.477653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:57.477653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:57.477653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:57.477653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:57.477653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:57.477653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:57.477653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:57.477653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:57.477653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:57.477653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:57.477653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:57.477653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:57.477653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:57.478639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:12:57.478639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:12:57.478639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:12:57.478639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:12:57.478639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:12:57.478639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:12:57.478639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:12:57.478639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:57.482802 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 368/711] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:57.483714 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c8255-3d87-410c-ae8b-44b3345e96fc None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:57.509779 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293005d-d443-45a3-bdc7-b23260ca19fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:57.510764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293005d-d443-45a3-bdc7-b23260ca19fa None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:57.511155 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293005d-d443-45a3-bdc7-b23260ca19fa None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:57.511534 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293005d-d443-45a3-bdc7-b23260ca19fa None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:57.512304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293005d-d443-45a3-bdc7-b23260ca19fa None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:57.513788 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293005d-d443-45a3-bdc7-b23260ca19fa None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:57.531071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-0293005d-d443-45a3-bdc7-b23260ca19fa None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:12:57.531071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:57.531071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:12:57.531071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:12:57.531071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:12:57.531071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:12:57.531071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:12:57.531071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:12:57.531071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:12:57.531071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:57.531071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:12:57.531071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:57.531071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:57.531071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:57.531071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:57.531071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:57.532323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:57.532323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:57.532323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:57.532323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:57.532323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:57.532323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:57.532323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:57.532323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:12:57.532323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:12:57.532323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:12:57.532323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:12:57.532323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:57.532323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:57.532323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:57.532323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:57.532323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:57.533374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:57.533374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:57.533374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:57.533374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:57.533374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:57.533374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:57.533374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:57.533374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:57.533374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:57.533374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:57.533374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:57.533374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:12:57.533374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:12:57.533374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:12:57.533374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:57.534551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 344/712] 10.222.0.78 () {56 vars in 1129 bytes} [Wed Jul 20 15:12:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:57.536190 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293005d-d443-45a3-bdc7-b23260ca19fa None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:57.593331 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:57.606508 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c8255-3d87-410c-ae8b-44b3345e96fc None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:57.651053 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b7a33-076d-47a6-8c87-37e0e5e681b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:57.652498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b7a33-076d-47a6-8c87-37e0e5e681b3 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:57.653059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b7a33-076d-47a6-8c87-37e0e5e681b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:57.653824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b7a33-076d-47a6-8c87-37e0e5e681b3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:57.654952 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b7a33-076d-47a6-8c87-37e0e5e681b3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:57.656259 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293005d-d443-45a3-bdc7-b23260ca19fa None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:57.656643 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b7a33-076d-47a6-8c87-37e0e5e681b3 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:57.675871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-da7b7a33-076d-47a6-8c87-37e0e5e681b3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:12:57.675871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:57.675871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:12:57.675871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:12:57.675871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:12:57.675871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:12:57.675871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:12:57.675871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:12:57.675871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:12:57.675871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:57.675871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:12:57.675871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:57.675871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:57.675871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:57.675871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:57.675871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:57.675871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:57.677116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:57.677116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:57.677116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:57.677116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:57.677116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:57.677116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:57.677116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:12:57.677116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:12:57.677116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:12:57.677116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:12:57.677116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:57.677116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:57.677116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:57.677116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:57.677116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:57.677116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:57.678154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:57.678154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:57.678154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:57.678154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:57.678154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:57.678154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:57.678154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:57.678154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:57.678154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:57.678154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:57.678154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:12:57.678154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:12:57.678154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:12:57.678154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:12:57.680912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 369/713] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:12:57 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:57.681964 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b7a33-076d-47a6-8c87-37e0e5e681b3 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:57.696678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157126f-b965-44fd-b4cb-98f85b34f0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:57.697408 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157126f-b965-44fd-b4cb-98f85b34f0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:57.697969 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157126f-b965-44fd-b4cb-98f85b34f0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:57.698220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157126f-b965-44fd-b4cb-98f85b34f0a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:57.699202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7157126f-b965-44fd-b4cb-98f85b34f0a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:57.699617 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7157126f-b965-44fd-b4cb-98f85b34f0a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:57.701217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7157126f-b965-44fd-b4cb-98f85b34f0a7 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:57.704388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:57.714830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 345/714] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:12:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:57.715953 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157126f-b965-44fd-b4cb-98f85b34f0a7 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:57.809548 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b7a33-076d-47a6-8c87-37e0e5e681b3 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:57.832394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157126f-b965-44fd-b4cb-98f85b34f0a7 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:57.848603 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802c52a-6f96-4769-a833-cb7cc2f5da05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:57.849369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802c52a-6f96-4769-a833-cb7cc2f5da05 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:57.849754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802c52a-6f96-4769-a833-cb7cc2f5da05 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:57.850134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802c52a-6f96-4769-a833-cb7cc2f5da05 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:57.850972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802c52a-6f96-4769-a833-cb7cc2f5da05 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:57.851372 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802c52a-6f96-4769-a833-cb7cc2f5da05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:57.852840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802c52a-6f96-4769-a833-cb7cc2f5da05 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:57.862176 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 370/715] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Jul 20 15:12:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:57.863321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802c52a-6f96-4769-a833-cb7cc2f5da05 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:57.866619 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:57.877040 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512ad99-8964-43f5-8a74-ffe5bab4358d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:57.878869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512ad99-8964-43f5-8a74-ffe5bab4358d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:57.879302 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512ad99-8964-43f5-8a74-ffe5bab4358d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:57.891520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512ad99-8964-43f5-8a74-ffe5bab4358d None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:57.891520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512ad99-8964-43f5-8a74-ffe5bab4358d None admin] RBAC: Adding query filter params (name=network) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:57.891520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512ad99-8964-43f5-8a74-ffe5bab4358d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:57.891520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512ad99-8964-43f5-8a74-ffe5bab4358d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:57.891520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 346/716] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Jul 20 15:12:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:57.891520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512ad99-8964-43f5-8a74-ffe5bab4358d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:57.965871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:57.992721 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802c52a-6f96-4769-a833-cb7cc2f5da05 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:58.018672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512ad99-8964-43f5-8a74-ffe5bab4358d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:58.032557 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7c694-a6d5-4e4a-8e95-afb125679a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:58.033538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7c694-a6d5-4e4a-8e95-afb125679a54 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:58.034038 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7c694-a6d5-4e4a-8e95-afb125679a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:58.034606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7c694-a6d5-4e4a-8e95-afb125679a54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:58.035632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7c694-a6d5-4e4a-8e95-afb125679a54 None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=a4a24d5a375945779fa626fae25ac237) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:58.036168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7c694-a6d5-4e4a-8e95-afb125679a54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:58.037810 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7c694-a6d5-4e4a-8e95-afb125679a54 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:58.044081 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 371/717] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:12:57 2022] GET /identity/v3/role_assignments?user.id=a4a24d5a375945779fa626fae25ac237&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=25516be1485c49538544ea8a596e7d17 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:58.045272 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7c694-a6d5-4e4a-8e95-afb125679a54 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:58.059599 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dccdc-dfba-4c69-ac66-a9a5c34a3bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:58.060265 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dccdc-dfba-4c69-ac66-a9a5c34a3bdc None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:58.060553 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dccdc-dfba-4c69-ac66-a9a5c34a3bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:58.060837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dccdc-dfba-4c69-ac66-a9a5c34a3bdc None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:58.061872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dccdc-dfba-4c69-ac66-a9a5c34a3bdc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:58.062189 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dccdc-dfba-4c69-ac66-a9a5c34a3bdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:58.063884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dccdc-dfba-4c69-ac66-a9a5c34a3bdc None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:58.072936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 347/718] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Jul 20 15:12:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:58.244448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7c694-a6d5-4e4a-8e95-afb125679a54 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:58.288113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74004c65-bad5-409e-bdcd-193a4adcff7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:58.288848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74004c65-bad5-409e-bdcd-193a4adcff7c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:58.289143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74004c65-bad5-409e-bdcd-193a4adcff7c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:58.289426 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74004c65-bad5-409e-bdcd-193a4adcff7c None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:58.290148 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74004c65-bad5-409e-bdcd-193a4adcff7c None admin] RBAC: Adding query filter params (type=network) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:58.290483 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74004c65-bad5-409e-bdcd-193a4adcff7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:58.293091 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74004c65-bad5-409e-bdcd-193a4adcff7c None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:58.299093 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 372/719] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Jul 20 15:12:58 2022] GET /identity/v3/services?type=network => generated 124 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:59.162134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc975d-a933-4861-9bbc-1b468fe95959 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:59.162677 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc975d-a933-4861-9bbc-1b468fe95959 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:59.162792 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc975d-a933-4861-9bbc-1b468fe95959 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:59.164270 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 348/720] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:59.169554 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6578e0-eb64-486f-a9e3-32dee8e055ad None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:59.170022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6578e0-eb64-486f-a9e3-32dee8e055ad None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:59.170486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6578e0-eb64-486f-a9e3-32dee8e055ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:59.182922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:59.188221 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-ca6578e0-eb64-486f-a9e3-32dee8e055ad None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:59.189506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ca6578e0-eb64-486f-a9e3-32dee8e055ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:12:59.257696 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 373/721] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:12:59.281394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc975d-a933-4861-9bbc-1b468fe95959 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:59.323647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:59.422598 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc975d-a933-4861-9bbc-1b468fe95959 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:59.468053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97335857-b821-49b7-b87f-66b2c633daf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:59.468939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97335857-b821-49b7-b87f-66b2c633daf6 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:59.469292 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97335857-b821-49b7-b87f-66b2c633daf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:59.469645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97335857-b821-49b7-b87f-66b2c633daf6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:59.480812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97335857-b821-49b7-b87f-66b2c633daf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:12:59.489632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-97335857-b821-49b7-b87f-66b2c633daf6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:59.489632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:59.489632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:59.489632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:59.489632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:59.489632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:59.489632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:59.489632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:59.489632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:59.489632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:59.489632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:59.489632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:59.489632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:12:59.489632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:12:59.489632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:12:59.489632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:12:59.490816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:12:59.490816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:12:59.490816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:12:59.490816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:12:59.490816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:59.490816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:59.490816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:59.490816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:59.490816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:59.490816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:59.490816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:59.490816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:59.490816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:59.490816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:59.490816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:59.490816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:59.491865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:59.491865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:59.491865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:59.491865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:59.491865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:12:59.491865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:12:59.491865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:12:59.491865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:12:59.491865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:12:59.491865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:12:59.491865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:12:59.491865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:59.492852 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 349/722] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:12:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:59.499014 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6578e0-eb64-486f-a9e3-32dee8e055ad None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:59.628883 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6578e0-eb64-486f-a9e3-32dee8e055ad None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:59.679075 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1dc01-662f-4183-ae5d-25b7978fb56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:59.679819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1dc01-662f-4183-ae5d-25b7978fb56f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:59.680172 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1dc01-662f-4183-ae5d-25b7978fb56f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:59.680518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1dc01-662f-4183-ae5d-25b7978fb56f None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:59.681347 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1dc01-662f-4183-ae5d-25b7978fb56f None admin] RBAC: Adding query filter params (name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:12:59.681726 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1dc01-662f-4183-ae5d-25b7978fb56f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:59.683213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1dc01-662f-4183-ae5d-25b7978fb56f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:59.689492 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:59.697094 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 374/723] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:12:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:12:59.702773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97335857-b821-49b7-b87f-66b2c633daf6 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:59.832918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:59.842658 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97335857-b821-49b7-b87f-66b2c633daf6 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:12:59.882023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb22339-9dd6-4455-8781-eea631f34535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:12:59.882951 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb22339-9dd6-4455-8781-eea631f34535 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:59.883342 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb22339-9dd6-4455-8781-eea631f34535 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:59.883689 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb22339-9dd6-4455-8781-eea631f34535 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:59.901373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb22339-9dd6-4455-8781-eea631f34535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:12:59.902944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb22339-9dd6-4455-8781-eea631f34535 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:12:59.927121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-1eb22339-9dd6-4455-8781-eea631f34535 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:59.927121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:12:59.927121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:12:59.927121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:12:59.927121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:12:59.927121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:12:59.927121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:12:59.927121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:12:59.927121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:12:59.927121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:12:59.927121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:12:59.927121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:12:59.927121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:12:59.927121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:12:59.927121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:12:59.928306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:12:59.928306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:12:59.928306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:12:59.928306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:12:59.928306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:12:59.928306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:12:59.928306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:12:59.928306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:12:59.928306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:12:59.928306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:12:59.928306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:12:59.928306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:12:59.928306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:12:59.928306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:12:59.928306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:12:59.928306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:12:59.928306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:12:59.929399 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:12:59.929399 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:12:59.929399 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:12:59.929399 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:12:59.930331 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa49c8d-854d-4dbf-9e4c-ddadb1e37a83 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:59.930629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa49c8d-854d-4dbf-9e4c-ddadb1e37a83 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:59.930912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa49c8d-854d-4dbf-9e4c-ddadb1e37a83 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:59.932532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 375/724] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:12:59.933664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 350/725] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:12:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:12:59.937278 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cabbf-c3d2-4e2a-86fd-a9f3a68dffbf None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:12:59.937571 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cabbf-c3d2-4e2a-86fd-a9f3a68dffbf None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:12:59.937853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cabbf-c3d2-4e2a-86fd-a9f3a68dffbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:12:59.939401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb22339-9dd6-4455-8781-eea631f34535 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:12:59.956804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6c1cabbf-c3d2-4e2a-86fd-a9f3a68dffbf None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:12:59.958305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6c1cabbf-c3d2-4e2a-86fd-a9f3a68dffbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:00.040219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 376/726] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:12:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:13:00.065883 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cabbf-c3d2-4e2a-86fd-a9f3a68dffbf None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:00.180488 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb22339-9dd6-4455-8781-eea631f34535 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:00.190330 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cabbf-c3d2-4e2a-86fd-a9f3a68dffbf None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:00.227686 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d4917-cc8e-4b69-9408-d25b93030881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:00.228729 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d4917-cc8e-4b69-9408-d25b93030881 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:00.229169 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d4917-cc8e-4b69-9408-d25b93030881 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:00.229586 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d4917-cc8e-4b69-9408-d25b93030881 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:00.241993 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d4917-cc8e-4b69-9408-d25b93030881 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:13:00.250219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-652d4917-cc8e-4b69-9408-d25b93030881 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:13:00.250219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:00.250219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:00.250219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:00.250219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:00.250219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:00.250219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:00.250219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:00.250219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:00.250219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:00.250219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:00.250219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:00.250219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:13:00.250219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:13:00.250219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:13:00.250219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:13:00.251361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:13:00.251361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:13:00.251361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:13:00.251361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:13:00.251361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:00.251361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:00.251361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:00.251361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:00.251361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:00.251361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:00.251361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:00.251361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:00.251361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:00.251361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:00.251361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:00.251361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:00.252693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:00.252693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:00.252693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:00.252693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:00.252693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:13:00.252693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:13:00.252693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:13:00.252693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:13:00.252693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:13:00.252693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:13:00.252693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:13:00.252693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:00.255277 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 377/727] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:13:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:00.258441 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:00.261464 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d4917-cc8e-4b69-9408-d25b93030881 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:00.282999 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e952d-c820-42c4-85be-66ce2e1410a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:00.284153 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e952d-c820-42c4-85be-66ce2e1410a3 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:00.285240 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e952d-c820-42c4-85be-66ce2e1410a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:00.285669 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e952d-c820-42c4-85be-66ce2e1410a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:00.286589 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e952d-c820-42c4-85be-66ce2e1410a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:00.287069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e952d-c820-42c4-85be-66ce2e1410a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:00.288609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e952d-c820-42c4-85be-66ce2e1410a3 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:00.308669 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 351/728] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:00.314295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e952d-c820-42c4-85be-66ce2e1410a3 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:00.318766 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:00.398402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d4917-cc8e-4b69-9408-d25b93030881 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:00.442370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063abe0-df2f-40a1-8dd8-4248b5475f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:00.442370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063abe0-df2f-40a1-8dd8-4248b5475f2e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:00.442370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063abe0-df2f-40a1-8dd8-4248b5475f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:00.442370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063abe0-df2f-40a1-8dd8-4248b5475f2e None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:00.442813 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063abe0-df2f-40a1-8dd8-4248b5475f2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:00.442813 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063abe0-df2f-40a1-8dd8-4248b5475f2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:00.442813 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063abe0-df2f-40a1-8dd8-4248b5475f2e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:00.458286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 378/729] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:13:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:00.458286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063abe0-df2f-40a1-8dd8-4248b5475f2e None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:00.542893 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e952d-c820-42c4-85be-66ce2e1410a3 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:00.557494 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:00.613556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a050b28-d99f-4f3f-9d8a-161732050821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:00.614391 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a050b28-d99f-4f3f-9d8a-161732050821 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:00.614672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a050b28-d99f-4f3f-9d8a-161732050821 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:00.615095 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a050b28-d99f-4f3f-9d8a-161732050821 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:00.623317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a050b28-d99f-4f3f-9d8a-161732050821 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:00.625040 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a050b28-d99f-4f3f-9d8a-161732050821 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:00.642312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-1a050b28-d99f-4f3f-9d8a-161732050821 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:13:00.642312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:00.642312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:00.642312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:00.642312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:00.642312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:00.642312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:00.642312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:00.642312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:00.642312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:00.642312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:00.642312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:00.642312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:13:00.642312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:13:00.642312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:13:00.642312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:00.643366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:00.644373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:00.644373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:00.644373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:13:00.644373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:13:00.644373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:13:00.644373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:13:00.644373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:13:00.644373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:13:00.644373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:13:00.644373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:00.644936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 352/730] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Jul 20 15:13:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:00.650569 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a050b28-d99f-4f3f-9d8a-161732050821 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:00.664877 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063abe0-df2f-40a1-8dd8-4248b5475f2e None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:00.719702 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:00.734822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4cb443-6c38-4438-8a74-fb10dafda01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:00.735622 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cb443-6c38-4438-8a74-fb10dafda01a None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:00.735902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cb443-6c38-4438-8a74-fb10dafda01a None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:00.736178 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cb443-6c38-4438-8a74-fb10dafda01a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:00.752620 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4cb443-6c38-4438-8a74-fb10dafda01a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:00.754412 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4cb443-6c38-4438-8a74-fb10dafda01a None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:00.773706 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a050b28-d99f-4f3f-9d8a-161732050821 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:00.777189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-9d4cb443-6c38-4438-8a74-fb10dafda01a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:00.777189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:00.777189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:00.777189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:00.777189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:00.777189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:00.777189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:00.777189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:00.777189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:00.777189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:00.777189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:00.777189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:00.777189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:00.777189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:00.777189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:00.778328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:00.778328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:00.778328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:00.778328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:00.778328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:00.778328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:00.778328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:00.778328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:00.778328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:00.778328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:00.778328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:00.778328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:00.778328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:00.778328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:00.778328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:00.778328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:00.778328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:00.779481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:00.779481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:00.779481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:00.779481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:00.782296 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 379/731] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:00.786246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5282d2-324c-4626-9c18-fcf6718726a1 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:00.786591 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5282d2-324c-4626-9c18-fcf6718726a1 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:00.786914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5282d2-324c-4626-9c18-fcf6718726a1 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:00.789979 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 380/732] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:00.789979 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5282d2-324c-4626-9c18-fcf6718726a1 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:00.818650 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95def086-aab7-4224-90ec-563b1a5a414d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:00.819476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95def086-aab7-4224-90ec-563b1a5a414d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:00.819864 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95def086-aab7-4224-90ec-563b1a5a414d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:00.820257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95def086-aab7-4224-90ec-563b1a5a414d None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:00.821196 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95def086-aab7-4224-90ec-563b1a5a414d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:00.821611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95def086-aab7-4224-90ec-563b1a5a414d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:00.823159 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95def086-aab7-4224-90ec-563b1a5a414d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:00.841006 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 353/733] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Jul 20 15:13:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:00.842423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38499a80-5a91-4451-9c7c-89500a2b05c7 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:00.842810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38499a80-5a91-4451-9c7c-89500a2b05c7 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:00.843187 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38499a80-5a91-4451-9c7c-89500a2b05c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:00.860155 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-38499a80-5a91-4451-9c7c-89500a2b05c7 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:00.861484 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-38499a80-5a91-4451-9c7c-89500a2b05c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:00.900270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5282d2-324c-4626-9c18-fcf6718726a1 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:00.901759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:00.928230 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 354/734] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:13:00.929629 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d665f45-ffc0-42c8-a20e-bb549d9bea15 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:00.929924 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d665f45-ffc0-42c8-a20e-bb549d9bea15 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:00.930202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d665f45-ffc0-42c8-a20e-bb549d9bea15 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:00.931477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 355/735] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:00.934397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d665f45-ffc0-42c8-a20e-bb549d9bea15 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:00.941541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9de03-66b2-455e-91eb-9c14dc59ebe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:00.942414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9de03-66b2-455e-91eb-9c14dc59ebe6 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:00.943002 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9de03-66b2-455e-91eb-9c14dc59ebe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:00.943338 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9de03-66b2-455e-91eb-9c14dc59ebe6 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:00.944118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9de03-66b2-455e-91eb-9c14dc59ebe6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:00.944489 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9de03-66b2-455e-91eb-9c14dc59ebe6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:00.945912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9de03-66b2-455e-91eb-9c14dc59ebe6 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:00.961992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 381/736] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:00.961992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4deacdc-bd97-410f-9caa-ab67be4aaa72 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:00.961992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4deacdc-bd97-410f-9caa-ab67be4aaa72 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:00.961992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4deacdc-bd97-410f-9caa-ab67be4aaa72 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:00.961992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 382/737] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:00.961992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00350408-f25e-4896-aa75-39436da43b4f None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:00.961992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00350408-f25e-4896-aa75-39436da43b4f None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:00.962660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00350408-f25e-4896-aa75-39436da43b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:00.978749 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-00350408-f25e-4896-aa75-39436da43b4f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:00.979857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-00350408-f25e-4896-aa75-39436da43b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:00.999618 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:01.045870 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 383/738] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:13:01.046714 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00350408-f25e-4896-aa75-39436da43b4f None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:01.052051 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d665f45-ffc0-42c8-a20e-bb549d9bea15 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:01.087505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cfd31-ba02-489f-8bae-69e329ef1a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:01.088162 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cfd31-ba02-489f-8bae-69e329ef1a40 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:01.088448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cfd31-ba02-489f-8bae-69e329ef1a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:01.088735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cfd31-ba02-489f-8bae-69e329ef1a40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:01.140226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:01.149286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00350408-f25e-4896-aa75-39436da43b4f None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:01.157558 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cfd31-ba02-489f-8bae-69e329ef1a40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:01.159042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cfd31-ba02-489f-8bae-69e329ef1a40 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:01.183047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4584a5e-ff23-4dc8-9809-692844f97b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:01.183709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4584a5e-ff23-4dc8-9809-692844f97b55 None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:01.183995 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4584a5e-ff23-4dc8-9809-692844f97b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:01.184267 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4584a5e-ff23-4dc8-9809-692844f97b55 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:01.184983 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4584a5e-ff23-4dc8-9809-692844f97b55 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:01.186362 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4584a5e-ff23-4dc8-9809-692844f97b55 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:01.206949 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:01.256183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 356/739] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:13:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:01.256931 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cfd31-ba02-489f-8bae-69e329ef1a40 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:01.294360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 384/740] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Jul 20 15:13:01 2022] POST /identity/v3/services => generated 240 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:13:01.295683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e76203-f074-4a97-bc0c-12b66e9cf06c None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:01.295975 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e76203-f074-4a97-bc0c-12b66e9cf06c None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:01.296256 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e76203-f074-4a97-bc0c-12b66e9cf06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:01.316443 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-12e76203-f074-4a97-bc0c-12b66e9cf06c None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:01.317724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-12e76203-f074-4a97-bc0c-12b66e9cf06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:01.414293 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 385/741] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:01 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 15:13:01.414293 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e76203-f074-4a97-bc0c-12b66e9cf06c None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:01.483349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cfd31-ba02-489f-8bae-69e329ef1a40 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:01.503687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:01.526334 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce0d38-73b1-4896-8448-771d30d515bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:01.526532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e76203-f074-4a97-bc0c-12b66e9cf06c None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:01.527287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce0d38-73b1-4896-8448-771d30d515bd None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:01.527623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce0d38-73b1-4896-8448-771d30d515bd None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:01.527957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce0d38-73b1-4896-8448-771d30d515bd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:01.537025 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce0d38-73b1-4896-8448-771d30d515bd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:01.538522 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce0d38-73b1-4896-8448-771d30d515bd None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:01.555211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-60ce0d38-73b1-4896-8448-771d30d515bd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:13:01.555211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:01.555211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:01.555211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:01.555211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:01.555211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:01.555211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:01.555211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:01.555211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:01.555211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:01.555211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:01.555211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:01.555211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:13:01.555211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:13:01.555211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:13:01.555211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:01.556300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:01.557262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:01.557262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:01.557262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:13:01.557262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:13:01.557262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:13:01.557262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:13:01.557262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:13:01.557262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:13:01.557262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:13:01.557262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:01.561528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 357/742] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:13:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:01.562561 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce0d38-73b1-4896-8448-771d30d515bd None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:01.577167 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:01.672971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce0d38-73b1-4896-8448-771d30d515bd None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:01.677686 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91915f1a-7dab-40d4-a404-86038bf17614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:01.678376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91915f1a-7dab-40d4-a404-86038bf17614 None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:01.678663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91915f1a-7dab-40d4-a404-86038bf17614 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:01.678938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91915f1a-7dab-40d4-a404-86038bf17614 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:01.679662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91915f1a-7dab-40d4-a404-86038bf17614 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:01.681052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91915f1a-7dab-40d4-a404-86038bf17614 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:01.707673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e962eec-0387-4cfa-8fe3-b4a4c837a671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:01.708598 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e962eec-0387-4cfa-8fe3-b4a4c837a671 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:01.708882 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e962eec-0387-4cfa-8fe3-b4a4c837a671 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:01.709158 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e962eec-0387-4cfa-8fe3-b4a4c837a671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:01.719022 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e962eec-0387-4cfa-8fe3-b4a4c837a671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:01.722382 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e962eec-0387-4cfa-8fe3-b4a4c837a671 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:01.738976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-0e962eec-0387-4cfa-8fe3-b4a4c837a671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:01.738976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:01.738976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:01.738976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:01.738976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:01.738976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:01.738976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:01.738976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:01.738976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:01.738976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:01.738976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:01.738976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:01.738976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:01.738976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:01.738976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:01.739930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:01.739930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:01.739930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:01.739930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:01.739930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:01.739930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:01.739930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:01.739930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:01.739930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:01.739930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:01.739930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:01.739930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:01.739930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:01.739930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:01.739930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:01.739930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:01.739930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:01.741148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:01.741148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:01.741148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:01.741148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:01.741148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 358/743] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:01.741148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e962eec-0387-4cfa-8fe3-b4a4c837a671 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:01.753702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 386/744] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Jul 20 15:13:01 2022] POST /identity/v3/services => generated 251 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:13:01.755171 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91915f1a-7dab-40d4-a404-86038bf17614 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:01.820878 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:01.877856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e962eec-0387-4cfa-8fe3-b4a4c837a671 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:01.896547 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91915f1a-7dab-40d4-a404-86038bf17614 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:01.924630 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8c226-634b-48e2-a020-506340609746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:01.926298 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8c226-634b-48e2-a020-506340609746 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:01.926773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8c226-634b-48e2-a020-506340609746 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:01.927150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8c226-634b-48e2-a020-506340609746 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:01.939753 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096b711-9824-4b20-a011-9896c8276985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:01.939753 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096b711-9824-4b20-a011-9896c8276985 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:01.940028 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096b711-9824-4b20-a011-9896c8276985 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:01.940210 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096b711-9824-4b20-a011-9896c8276985 None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:01.941622 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096b711-9824-4b20-a011-9896c8276985 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:01.941936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096b711-9824-4b20-a011-9896c8276985 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:01.943353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8c226-634b-48e2-a020-506340609746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:13:01.943699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096b711-9824-4b20-a011-9896c8276985 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:01.945448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:01.953527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-02a8c226-634b-48e2-a020-506340609746 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:13:01.953527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:01.953527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:01.953527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:01.953527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:01.953527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:01.953527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:01.953527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:01.953527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:01.953527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:01.953527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:01.953527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:01.953527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:13:01.953527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:13:01.953527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:13:01.953527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:13:02.160027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:13:02.160027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:13:02.160027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:13:02.160027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:13:02.160027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:02.160027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:02.160027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:02.160027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:02.160027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:02.160027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:02.160027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:02.160027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:02.160027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:02.160027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:02.160027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:02.160027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:02.161480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:02.161480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:02.161480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:02.161480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:02.161480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:13:02.161480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:13:02.161480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:13:02.161480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:13:02.161480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:13:02.161480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:13:02.161480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:13:02.161480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:02.161480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 359/745] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:13:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:02.161480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8c226-634b-48e2-a020-506340609746 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:02.162482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 387/746] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Jul 20 15:13:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:02.162482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096b711-9824-4b20-a011-9896c8276985 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:02.162637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8c226-634b-48e2-a020-506340609746 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:02.162719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096b711-9824-4b20-a011-9896c8276985 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:02.162888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:02.162888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6745ae8-8431-48d9-aefb-ef4437c68812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:02.163057 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85003d0-10e5-4c71-9854-304f4107b428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:02.163150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6745ae8-8431-48d9-aefb-ef4437c68812 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:02.163242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85003d0-10e5-4c71-9854-304f4107b428 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:02.163325 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6745ae8-8431-48d9-aefb-ef4437c68812 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:02.163405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85003d0-10e5-4c71-9854-304f4107b428 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:02.163492 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6745ae8-8431-48d9-aefb-ef4437c68812 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:02.163574 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85003d0-10e5-4c71-9854-304f4107b428 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:02.163644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6745ae8-8431-48d9-aefb-ef4437c68812 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:02.163733 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85003d0-10e5-4c71-9854-304f4107b428 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:02.163816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6745ae8-8431-48d9-aefb-ef4437c68812 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:02.163897 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85003d0-10e5-4c71-9854-304f4107b428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:02.163978 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6745ae8-8431-48d9-aefb-ef4437c68812 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:02.164059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85003d0-10e5-4c71-9854-304f4107b428 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:02.164059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 388/747] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:13:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:02.164059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85003d0-10e5-4c71-9854-304f4107b428 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:02.164293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 360/748] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:02.164293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6745ae8-8431-48d9-aefb-ef4437c68812 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:02.235457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:02.248424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85003d0-10e5-4c71-9854-304f4107b428 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:02.250800 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6745ae8-8431-48d9-aefb-ef4437c68812 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:02.289348 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a4f9b-030b-42d4-a238-874e5167ae2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:02.290510 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a4f9b-030b-42d4-a238-874e5167ae2e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:02.290745 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a4f9b-030b-42d4-a238-874e5167ae2e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:02.291116 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a4f9b-030b-42d4-a238-874e5167ae2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:02.299715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79f319-93d4-4675-9cce-2d276041e0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:02.301412 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79f319-93d4-4675-9cce-2d276041e0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:02.301851 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79f319-93d4-4675-9cce-2d276041e0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:02.302408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79f319-93d4-4675-9cce-2d276041e0d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:02.304174 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a4f9b-030b-42d4-a238-874e5167ae2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:02.305814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a4f9b-030b-42d4-a238-874e5167ae2e None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:02.325358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-e21a4f9b-030b-42d4-a238-874e5167ae2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:02.325358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:02.325358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:02.325358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:02.325358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:02.325358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:02.325358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:02.325358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:02.325358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:02.325358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:02.325358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:02.325358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:02.325358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:02.325358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:02.325358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:02.326365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:02.326365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:02.326365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:02.326365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:02.326365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:02.326365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:02.326365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:02.326365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:02.326365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:02.326365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:02.326365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:02.326365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:02.326365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:02.326365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:02.326365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:02.326365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:02.326365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:02.327277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:02.327277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:02.327277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:02.327277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:02.331912 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 361/749] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:02.332889 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a4f9b-030b-42d4-a238-874e5167ae2e None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:02.371475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:02.392184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79f319-93d4-4675-9cce-2d276041e0d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:02.395023 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79f319-93d4-4675-9cce-2d276041e0d5 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:02.454384 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a4f9b-030b-42d4-a238-874e5167ae2e None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:02.465774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:02.503784 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa586ac-fc60-40ba-911d-8a370a300258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:02.504962 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa586ac-fc60-40ba-911d-8a370a300258 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:02.505450 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa586ac-fc60-40ba-911d-8a370a300258 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:02.505925 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa586ac-fc60-40ba-911d-8a370a300258 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:02.515909 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa586ac-fc60-40ba-911d-8a370a300258 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:02.517875 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa586ac-fc60-40ba-911d-8a370a300258 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:02.533667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-9fa586ac-fc60-40ba-911d-8a370a300258 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:02.533667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:02.533667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:02.533667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:02.533667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:02.533667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:02.533667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:02.533667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:02.533667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:02.533667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:02.533667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:02.533667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:02.533667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:13:02.533667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:13:02.533667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:13:02.535195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:13:02.535195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:02.535195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:02.535195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:02.535195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:02.535195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:02.535195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:02.535195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:02.535195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:02.535195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:02.535195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:02.535195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:02.535195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:02.535195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:02.535195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:02.535195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:02.535195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:02.536665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:13:02.536665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:13:02.536665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:13:02.536665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:13:02.536665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:13:02.536665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:13:02.536665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:02.536665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:02.537113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 389/750] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:13:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:02.543360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79f319-93d4-4675-9cce-2d276041e0d5 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:02.543638 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 362/751] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:02.544811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa586ac-fc60-40ba-911d-8a370a300258 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:02.648715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79f319-93d4-4675-9cce-2d276041e0d5 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:02.664529 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:02.670816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa586ac-fc60-40ba-911d-8a370a300258 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:02.697252 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564cf3e-c4b7-4cfc-9eb9-b60d8a78f795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:02.698085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564cf3e-c4b7-4cfc-9eb9-b60d8a78f795 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:02.698538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564cf3e-c4b7-4cfc-9eb9-b60d8a78f795 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:02.698827 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564cf3e-c4b7-4cfc-9eb9-b60d8a78f795 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:02.699559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564cf3e-c4b7-4cfc-9eb9-b60d8a78f795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:02.699873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564cf3e-c4b7-4cfc-9eb9-b60d8a78f795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:02.702010 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564cf3e-c4b7-4cfc-9eb9-b60d8a78f795 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:02.710224 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e6f2d-d54e-4138-9021-39b5d206e11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:02.711380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e6f2d-d54e-4138-9021-39b5d206e11c None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:02.711869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e6f2d-d54e-4138-9021-39b5d206e11c None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:02.712432 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e6f2d-d54e-4138-9021-39b5d206e11c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:02.721503 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 390/752] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:02.722967 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564cf3e-c4b7-4cfc-9eb9-b60d8a78f795 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:02.726100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e6f2d-d54e-4138-9021-39b5d206e11c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:02.728084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e6f2d-d54e-4138-9021-39b5d206e11c None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:02.744842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:02.750833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-820e6f2d-d54e-4138-9021-39b5d206e11c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:02.750833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:02.750833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:02.750833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:02.750833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:02.750833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:02.750833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:02.750833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:02.750833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:02.750833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:02.750833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:02.750833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:02.750833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:02.750833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:02.750833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:02.751774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:02.751774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:02.751774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:02.751774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:02.751774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:02.751774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:02.751774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:02.751774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:02.751774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:02.751774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:02.751774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:02.751774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:02.751774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:02.751774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:02.751774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:02.751774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:02.751774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:02.753007 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:02.753007 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:02.753007 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:02.753007 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:02.759548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 363/753] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:02.760614 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e6f2d-d54e-4138-9021-39b5d206e11c None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:02.854540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564cf3e-c4b7-4cfc-9eb9-b60d8a78f795 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:02.871086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e6f2d-d54e-4138-9021-39b5d206e11c None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:02.902536 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436e7a6-cf53-4512-b114-6b86ebae0997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:02.903257 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436e7a6-cf53-4512-b114-6b86ebae0997 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:02.903700 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436e7a6-cf53-4512-b114-6b86ebae0997 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:02.904158 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436e7a6-cf53-4512-b114-6b86ebae0997 None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:02.904957 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436e7a6-cf53-4512-b114-6b86ebae0997 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:02.905273 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436e7a6-cf53-4512-b114-6b86ebae0997 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:02.907560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436e7a6-cf53-4512-b114-6b86ebae0997 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:02.911891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1375cc-72f4-4795-9085-ed091c60f19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:02.912913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1375cc-72f4-4795-9085-ed091c60f19e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:02.913416 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1375cc-72f4-4795-9085-ed091c60f19e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:02.913891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1375cc-72f4-4795-9085-ed091c60f19e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:02.925309 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 391/754] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:13:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:02.925309 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436e7a6-cf53-4512-b114-6b86ebae0997 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:02.925558 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1375cc-72f4-4795-9085-ed091c60f19e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:02.925920 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1375cc-72f4-4795-9085-ed091c60f19e None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:02.946410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-ae1375cc-72f4-4795-9085-ed091c60f19e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:13:02.946410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:02.946410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:02.946410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:02.946410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:02.946410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:02.946410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:02.946410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:02.946410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:02.946410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:02.946410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:02.946410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:02.946410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:13:02.946410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:13:02.946410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:13:02.946410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:02.947431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:02.948429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:02.948429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:02.948429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:13:02.948429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:13:02.948429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:13:02.948429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:13:02.948429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:13:02.948429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:13:02.948429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:13:02.948429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:02.951504 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 364/755] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Jul 20 15:13:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:02.952640 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1375cc-72f4-4795-9085-ed091c60f19e None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:02.964576 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:03.026632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:03.060088 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436e7a6-cf53-4512-b114-6b86ebae0997 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:03.068001 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1375cc-72f4-4795-9085-ed091c60f19e None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:03.108731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d673f87-292c-41e5-bca6-ce50f0470bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:03.109401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d673f87-292c-41e5-bca6-ce50f0470bda None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:03.109735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d673f87-292c-41e5-bca6-ce50f0470bda None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:03.110009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d673f87-292c-41e5-bca6-ce50f0470bda None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:03.110768 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d673f87-292c-41e5-bca6-ce50f0470bda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:03.111074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d673f87-292c-41e5-bca6-ce50f0470bda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:03.113119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d673f87-292c-41e5-bca6-ce50f0470bda None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:03.113334 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e2bb6-05bc-4532-83d9-03b6b15348d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:03.114206 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e2bb6-05bc-4532-83d9-03b6b15348d2 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:03.114709 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e2bb6-05bc-4532-83d9-03b6b15348d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:03.115174 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e2bb6-05bc-4532-83d9-03b6b15348d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:03.116228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e2bb6-05bc-4532-83d9-03b6b15348d2 None admin] RBAC: Adding query filter params (role.id=d820b1d3fa4144aca575c04645c099bc, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=4e75a82cc7e94702ba0080fc65d6eef3) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:03.116737 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e2bb6-05bc-4532-83d9-03b6b15348d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:03.118378 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e2bb6-05bc-4532-83d9-03b6b15348d2 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:03.132075 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 392/757] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:03.132075 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d673f87-292c-41e5-bca6-ce50f0470bda None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:03.132392 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 365/757] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:13:02 2022] GET /identity/v3/role_assignments?user.id=4e75a82cc7e94702ba0080fc65d6eef3&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=d820b1d3fa4144aca575c04645c099bc => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:03.139737 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e2bb6-05bc-4532-83d9-03b6b15348d2 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:03.248392 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:03.286441 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e2bb6-05bc-4532-83d9-03b6b15348d2 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:03.327428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa6d7c-8742-4b03-b20d-9efa683d2150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:03.329401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa6d7c-8742-4b03-b20d-9efa683d2150 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:03.329832 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa6d7c-8742-4b03-b20d-9efa683d2150 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:03.330254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa6d7c-8742-4b03-b20d-9efa683d2150 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:03.339804 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa6d7c-8742-4b03-b20d-9efa683d2150 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:03.342104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa6d7c-8742-4b03-b20d-9efa683d2150 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:03.362229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-46fa6d7c-8742-4b03-b20d-9efa683d2150 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:03.362229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:03.362229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:03.362229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:03.362229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:03.362229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:03.362229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:03.362229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:03.362229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:03.362229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:03.362229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:03.362229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:03.362229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:13:03.362229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:13:03.362229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:13:03.363876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:13:03.363876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:03.363876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:03.363876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:03.363876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:03.363876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:03.363876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:03.363876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:03.363876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:03.363876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:03.363876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:03.363876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:03.363876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:03.363876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:03.363876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:03.363876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:03.363876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:03.365016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:13:03.365016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:13:03.365016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:13:03.365016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:13:03.365016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:13:03.365016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:13:03.365016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:03.365016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:03.366727 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d673f87-292c-41e5-bca6-ce50f0470bda None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:03.367701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 366/758] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:03.374753 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa6d7c-8742-4b03-b20d-9efa683d2150 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:03.428946 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c28705-1afb-4095-9e39-5986e9cdca7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:03.429631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c28705-1afb-4095-9e39-5986e9cdca7e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:03.429911 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c28705-1afb-4095-9e39-5986e9cdca7e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:03.430186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c28705-1afb-4095-9e39-5986e9cdca7e None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:03.431296 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c28705-1afb-4095-9e39-5986e9cdca7e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:03.431421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c28705-1afb-4095-9e39-5986e9cdca7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:03.432782 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c28705-1afb-4095-9e39-5986e9cdca7e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:03.434684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:03.451377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 393/759] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Jul 20 15:13:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:03.452723 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c070e4d-d993-4a9b-b989-ac822b6c0238 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:03.452953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c070e4d-d993-4a9b-b989-ac822b6c0238 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:03.453247 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c070e4d-d993-4a9b-b989-ac822b6c0238 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:03.454726 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 394/760] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:03.459366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c070e4d-d993-4a9b-b989-ac822b6c0238 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:03.500185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa6d7c-8742-4b03-b20d-9efa683d2150 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:03.541501 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:03.549209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510b770-1e94-40fd-b9dd-0ca5aca482d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:03.550631 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510b770-1e94-40fd-b9dd-0ca5aca482d8 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:03.556192 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510b770-1e94-40fd-b9dd-0ca5aca482d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:03.556192 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510b770-1e94-40fd-b9dd-0ca5aca482d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:03.556192 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a510b770-1e94-40fd-b9dd-0ca5aca482d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:03.556192 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a510b770-1e94-40fd-b9dd-0ca5aca482d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:03.556192 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a510b770-1e94-40fd-b9dd-0ca5aca482d8 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:03.563130 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c070e4d-d993-4a9b-b989-ac822b6c0238 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:03.569753 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 367/761] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:13:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:03.571074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3842ad8-ed3a-4457-8e1b-c66b4d08dbcc None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:03.571834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3842ad8-ed3a-4457-8e1b-c66b4d08dbcc None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:03.572117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3842ad8-ed3a-4457-8e1b-c66b4d08dbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:03.594248 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-c3842ad8-ed3a-4457-8e1b-c66b4d08dbcc None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:03.595928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-c3842ad8-ed3a-4457-8e1b-c66b4d08dbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:03.601042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8508a7-c179-4cee-9c97-88d9ee7f0fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:03.601888 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8508a7-c179-4cee-9c97-88d9ee7f0fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:03.602173 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8508a7-c179-4cee-9c97-88d9ee7f0fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:03.602486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8508a7-c179-4cee-9c97-88d9ee7f0fb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:03.613689 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8508a7-c179-4cee-9c97-88d9ee7f0fb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:03.615376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8508a7-c179-4cee-9c97-88d9ee7f0fb1 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:03.634604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-2b8508a7-c179-4cee-9c97-88d9ee7f0fb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:03.634604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:03.634604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:03.634604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:03.634604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:03.634604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:03.634604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:03.634604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:03.634604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:03.634604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:03.634604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:03.634604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:03.634604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:03.634604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:03.634604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:03.635581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:03.635581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:03.635581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:03.635581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:03.635581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:03.635581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:03.635581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:03.635581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:03.635581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:03.635581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:03.635581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:03.635581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:03.635581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:03.635581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:03.635581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:03.635581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:03.635581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:03.636488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:03.636488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:03.636488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:03.636488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:03.640741 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 395/762] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:03.641733 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8508a7-c179-4cee-9c97-88d9ee7f0fb1 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:03.679149 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 368/763] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:03.680040 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3842ad8-ed3a-4457-8e1b-c66b4d08dbcc None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:03.702173 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:03.778185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8508a7-c179-4cee-9c97-88d9ee7f0fb1 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:03.800544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3842ad8-ed3a-4457-8e1b-c66b4d08dbcc None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:03.822148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:03.823613 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c30f0f-c612-4a3b-bc9c-6883acf9d039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:03.824294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c30f0f-c612-4a3b-bc9c-6883acf9d039 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:03.824582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c30f0f-c612-4a3b-bc9c-6883acf9d039 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:03.824864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c30f0f-c612-4a3b-bc9c-6883acf9d039 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:03.825975 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c30f0f-c612-4a3b-bc9c-6883acf9d039 None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=ed0d719c0eb04a3b8a228c6ce70fdf98) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:03.826352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c30f0f-c612-4a3b-bc9c-6883acf9d039 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:03.828401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c30f0f-c612-4a3b-bc9c-6883acf9d039 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:03.835537 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 396/764] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:13:03 2022] GET /identity/v3/role_assignments?user.id=ed0d719c0eb04a3b8a228c6ce70fdf98&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=25516be1485c49538544ea8a596e7d17 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:03.836332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c30f0f-c612-4a3b-bc9c-6883acf9d039 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:03.838799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27954196-85e0-4f02-90cc-35a5ee37cb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:03.839810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27954196-85e0-4f02-90cc-35a5ee37cb9f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:03.840357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27954196-85e0-4f02-90cc-35a5ee37cb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:03.840802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27954196-85e0-4f02-90cc-35a5ee37cb9f None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:03.841923 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27954196-85e0-4f02-90cc-35a5ee37cb9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:03.842428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27954196-85e0-4f02-90cc-35a5ee37cb9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:03.844277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27954196-85e0-4f02-90cc-35a5ee37cb9f None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:03.858486 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 369/765] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:03.865154 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27954196-85e0-4f02-90cc-35a5ee37cb9f None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:03.967059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c30f0f-c612-4a3b-bc9c-6883acf9d039 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:03.980355 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:03.984601 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27954196-85e0-4f02-90cc-35a5ee37cb9f None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:04.012967 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cb4c1-295a-4922-8504-4d919eb72d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:04.014176 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cb4c1-295a-4922-8504-4d919eb72d10 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:04.014575 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cb4c1-295a-4922-8504-4d919eb72d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:04.014823 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cb4c1-295a-4922-8504-4d919eb72d10 None admin] PATH_INFO: `/v3/services/network` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:04.016189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2cb4c1-295a-4922-8504-4d919eb72d10 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:04.018387 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2cb4c1-295a-4922-8504-4d919eb72d10 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:04.025268 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711daaac-1b7e-4f52-8f39-df4c86bb41e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:04.026301 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711daaac-1b7e-4f52-8f39-df4c86bb41e4 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:04.026732 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711daaac-1b7e-4f52-8f39-df4c86bb41e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:04.027154 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711daaac-1b7e-4f52-8f39-df4c86bb41e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:04.033393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-af2cb4c1-295a-4922-8504-4d919eb72d10 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:13:04.033393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:04.033393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:04.033393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:04.033393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:04.033393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:04.033393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:04.033393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:04.033393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:13:04.033393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:04.033393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:04.033393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:04.033393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:04.033393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:04.033393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:04.033393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:04.033393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:04.035188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:04.035188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:04.035188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:04.035188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:04.035188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:04.035188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:04.035188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:04.035188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:04.035188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:04.035188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:04.035188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:04.035188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:04.035188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:04.035188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:04.035188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:04.035188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:04.036253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:04.036253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:04.036253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:04.036253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:04.036253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:04.036253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:04.036253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:04.036253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:04.036253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:04.036253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:04.036253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:04.036253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:04.036253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:13:04.036253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:04.037340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 397/766] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:03 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:04.043586 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cb4c1-295a-4922-8504-4d919eb72d10 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:04.043927 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711daaac-1b7e-4f52-8f39-df4c86bb41e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:04.045604 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711daaac-1b7e-4f52-8f39-df4c86bb41e4 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:04.061780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-711daaac-1b7e-4f52-8f39-df4c86bb41e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:04.061780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:04.061780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:04.061780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:04.061780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:04.061780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:04.061780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:04.061780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:04.061780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:04.061780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:04.061780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:04.061780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:04.061780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:13:04.061780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:13:04.061780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:13:04.062773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:13:04.062773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:04.062773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:04.062773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:04.062773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:04.062773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:04.062773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:04.062773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:04.062773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:04.062773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:04.062773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:04.062773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:04.062773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:04.062773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:04.062773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:04.062773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:04.062773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:04.063718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:13:04.063718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:13:04.063718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:13:04.063718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:13:04.063718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:13:04.063718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:13:04.063718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:04.063718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:04.066418 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 370/767] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:04.072405 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711daaac-1b7e-4f52-8f39-df4c86bb41e4 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:04.122490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:04.157200 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cb4c1-295a-4922-8504-4d919eb72d10 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:04.195165 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711daaac-1b7e-4f52-8f39-df4c86bb41e4 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:04.195873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa4edc-9d43-4400-a5e8-c2b4e92a4ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:04.196642 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa4edc-9d43-4400-a5e8-c2b4e92a4ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:04.197018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa4edc-9d43-4400-a5e8-c2b4e92a4ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:04.197370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa4edc-9d43-4400-a5e8-c2b4e92a4ad0 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:04.198152 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa4edc-9d43-4400-a5e8-c2b4e92a4ad0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:04.198553 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa4edc-9d43-4400-a5e8-c2b4e92a4ad0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:04.200000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa4edc-9d43-4400-a5e8-c2b4e92a4ad0 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:04.205107 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 398/768] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Jul 20 15:13:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:04.207060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec9396-fa93-49f5-a602-689326ea1bba None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:04.207416 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec9396-fa93-49f5-a602-689326ea1bba None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:04.207766 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec9396-fa93-49f5-a602-689326ea1bba None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:04.209293 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 399/769] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:04.211856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec9396-fa93-49f5-a602-689326ea1bba None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:04.240573 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad2685-fcd5-4821-84eb-4c5234e8fa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:04.241249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad2685-fcd5-4821-84eb-4c5234e8fa09 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:04.241816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad2685-fcd5-4821-84eb-4c5234e8fa09 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:04.242099 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad2685-fcd5-4821-84eb-4c5234e8fa09 None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:04.243183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad2685-fcd5-4821-84eb-4c5234e8fa09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:04.243532 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad2685-fcd5-4821-84eb-4c5234e8fa09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:04.245654 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad2685-fcd5-4821-84eb-4c5234e8fa09 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:04.266616 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:04.285390 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 371/770] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:13:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:04.286459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43699735-7bb8-4c40-bcc3-5ad8cb91941f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:04.286751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43699735-7bb8-4c40-bcc3-5ad8cb91941f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:04.287282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43699735-7bb8-4c40-bcc3-5ad8cb91941f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:04.322296 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-43699735-7bb8-4c40-bcc3-5ad8cb91941f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:04.323482 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-43699735-7bb8-4c40-bcc3-5ad8cb91941f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:04.380783 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec9396-fa93-49f5-a602-689326ea1bba None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:04.426239 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec9425-9404-4b27-b22e-efe3dd5926fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:04.427166 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec9425-9404-4b27-b22e-efe3dd5926fa None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:04.427502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec9425-9404-4b27-b22e-efe3dd5926fa None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:04.427782 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec9425-9404-4b27-b22e-efe3dd5926fa None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:04.428495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec9425-9404-4b27-b22e-efe3dd5926fa None admin] RBAC: Adding query filter params (type=network) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:04.428801 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec9425-9404-4b27-b22e-efe3dd5926fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:04.430176 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec9425-9404-4b27-b22e-efe3dd5926fa None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:04.444174 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 400/771] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Jul 20 15:13:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:04.445045 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec9425-9404-4b27-b22e-efe3dd5926fa None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:04.534393 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 372/772] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:04.535670 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43699735-7bb8-4c40-bcc3-5ad8cb91941f None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:04.592568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:04.644553 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:04.651485 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec9425-9404-4b27-b22e-efe3dd5926fa None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:04.732601 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43699735-7bb8-4c40-bcc3-5ad8cb91941f None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:04.734086 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2124944-2324-474a-b031-ae125b1b3ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:04.735376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2124944-2324-474a-b031-ae125b1b3ae2 None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:04.736422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2124944-2324-474a-b031-ae125b1b3ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:04.738097 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2124944-2324-474a-b031-ae125b1b3ae2 None admin] PATH_INFO: `/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/a4a24d5a375945779fa626fae25ac237/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:04.741368 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2124944-2324-474a-b031-ae125b1b3ae2 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdae1cb71dd449b8a18062b7b52889e4, user_id=a4a24d5a375945779fa626fae25ac237, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:04.746441 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2124944-2324-474a-b031-ae125b1b3ae2 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:04.783606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247fb33-6d27-4a19-b704-f111bbc950cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:04.784340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247fb33-6d27-4a19-b704-f111bbc950cf None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:04.784626 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247fb33-6d27-4a19-b704-f111bbc950cf None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:04.784899 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247fb33-6d27-4a19-b704-f111bbc950cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:04.785738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247fb33-6d27-4a19-b704-f111bbc950cf None admin] RBAC: Adding query filter params (interface=public, service_id=5be0800e679a407180ad6fcce6a73bb7, region_id=RegionOne) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:04.786006 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247fb33-6d27-4a19-b704-f111bbc950cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:04.787421 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247fb33-6d27-4a19-b704-f111bbc950cf None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:04.800988 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 373/773] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Jul 20 15:13:04 2022] GET /identity/v3/endpoints?service_id=5be0800e679a407180ad6fcce6a73bb7&interface=public®ion_id=RegionOne => generated 194 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:04.800988 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247fb33-6d27-4a19-b704-f111bbc950cf None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:04.841618 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 401/774] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:13:04 2022] PUT /identity/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/a4a24d5a375945779fa626fae25ac237/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:13:04.841618 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2124944-2324-474a-b031-ae125b1b3ae2 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:04.969871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:04.981369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247fb33-6d27-4a19-b704-f111bbc950cf None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:05.035255 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2124944-2324-474a-b031-ae125b1b3ae2 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:05.053006 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d783a-0b26-4ee6-87fd-0acab3c6711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:05.053742 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d783a-0b26-4ee6-87fd-0acab3c6711f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:05.054022 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d783a-0b26-4ee6-87fd-0acab3c6711f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:05.054312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d783a-0b26-4ee6-87fd-0acab3c6711f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:05.054979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d783a-0b26-4ee6-87fd-0acab3c6711f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:05.056336 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d783a-0b26-4ee6-87fd-0acab3c6711f None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:05.075366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-333d783a-0b26-4ee6-87fd-0acab3c6711f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:13:05.075366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:05.075366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:05.075366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:05.075366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:05.075366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:05.075366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:05.075366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:05.075366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:13:05.075366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:05.075366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:05.075366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:05.075366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:05.075366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:05.075366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:05.075366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:05.076556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:05.076556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:05.076556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:05.076556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:05.076556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:05.076556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:05.076556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:05.076556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:05.076556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:05.076556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:05.076556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:05.076556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:05.076556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:05.076556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:05.076556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:05.076556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:05.077653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:05.077653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:05.077653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:05.077653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:05.077653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:05.077653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:05.077653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:05.077653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:05.077653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:05.077653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:05.077653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:05.077653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:05.077653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:05.077653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:13:05.077653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:05.077653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 374/775] 10.222.0.78 () {56 vars in 1129 bytes} [Wed Jul 20 15:13:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:05.083778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d783a-0b26-4ee6-87fd-0acab3c6711f None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:05.104289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793b937-47bf-44aa-a9be-b732a6ed69cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:05.105577 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793b937-47bf-44aa-a9be-b732a6ed69cc None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:05.106253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793b937-47bf-44aa-a9be-b732a6ed69cc None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:05.106704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793b937-47bf-44aa-a9be-b732a6ed69cc None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:05.107957 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793b937-47bf-44aa-a9be-b732a6ed69cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:05.109562 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793b937-47bf-44aa-a9be-b732a6ed69cc None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:05.122444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 402/776] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Jul 20 15:13:04 2022] GET /identity/v3/services => generated 772 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:05.224624 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d783a-0b26-4ee6-87fd-0acab3c6711f None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:05.278724 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809fb0b-1f1b-4682-8bd2-d5feb21a9c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:05.279410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809fb0b-1f1b-4682-8bd2-d5feb21a9c2d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:05.279690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809fb0b-1f1b-4682-8bd2-d5feb21a9c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:05.279962 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809fb0b-1f1b-4682-8bd2-d5feb21a9c2d None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:05.280685 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809fb0b-1f1b-4682-8bd2-d5feb21a9c2d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:05.280989 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809fb0b-1f1b-4682-8bd2-d5feb21a9c2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:05.282398 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809fb0b-1f1b-4682-8bd2-d5feb21a9c2d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:05.289311 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 375/777] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Jul 20 15:13:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:05.295037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793b937-47bf-44aa-a9be-b732a6ed69cc None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:05.332242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:05.408265 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793b937-47bf-44aa-a9be-b732a6ed69cc None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:05.445682 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c831a8-b5ef-4bb1-898e-6ff15839e253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:05.446392 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c831a8-b5ef-4bb1-898e-6ff15839e253 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:05.446674 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c831a8-b5ef-4bb1-898e-6ff15839e253 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:05.446950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c831a8-b5ef-4bb1-898e-6ff15839e253 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:05.447666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c831a8-b5ef-4bb1-898e-6ff15839e253 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:05.447966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c831a8-b5ef-4bb1-898e-6ff15839e253 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:05.449340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c831a8-b5ef-4bb1-898e-6ff15839e253 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:05.454738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 403/778] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Jul 20 15:13:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:05.460468 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809fb0b-1f1b-4682-8bd2-d5feb21a9c2d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:05.502654 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:05.575524 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78091265-1966-476b-bfbc-850a9d3512e1 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:05.575878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78091265-1966-476b-bfbc-850a9d3512e1 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:05.576089 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78091265-1966-476b-bfbc-850a9d3512e1 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:05.577679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 404/779] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:05.583524 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f42db-f1ee-49d1-bfd6-4b384b03475e None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:05.583749 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f42db-f1ee-49d1-bfd6-4b384b03475e None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:05.584023 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f42db-f1ee-49d1-bfd6-4b384b03475e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:05.614053 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-4c0f42db-f1ee-49d1-bfd6-4b384b03475e None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:05.615306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-4c0f42db-f1ee-49d1-bfd6-4b384b03475e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:05.633407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809fb0b-1f1b-4682-8bd2-d5feb21a9c2d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:05.671079 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1b0ce-bc37-4608-a212-e6e34ae4913d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:05.672066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1b0ce-bc37-4608-a212-e6e34ae4913d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:05.672526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1b0ce-bc37-4608-a212-e6e34ae4913d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:05.672984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1b0ce-bc37-4608-a212-e6e34ae4913d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:05.673956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1b0ce-bc37-4608-a212-e6e34ae4913d None admin] RBAC: Adding query filter params (interface=public, service_id=f9af2578128a4314af56f725d8fa1767, region_id=RegionOne) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:05.674466 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1b0ce-bc37-4608-a212-e6e34ae4913d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:05.676854 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1b0ce-bc37-4608-a212-e6e34ae4913d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:05.683039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 376/780] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Jul 20 15:13:05 2022] GET /identity/v3/endpoints?service_id=f9af2578128a4314af56f725d8fa1767&interface=public®ion_id=RegionOne => generated 194 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:05.688534 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1b0ce-bc37-4608-a212-e6e34ae4913d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:05.759614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 405/781] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:05.781578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:05.793482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f42db-f1ee-49d1-bfd6-4b384b03475e None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:05.800360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1b0ce-bc37-4608-a212-e6e34ae4913d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:05.831460 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:05.844532 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12606c-992e-4df7-8e5d-65becf19818b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:05.844532 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12606c-992e-4df7-8e5d-65becf19818b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:05.844532 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12606c-992e-4df7-8e5d-65becf19818b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:05.844532 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12606c-992e-4df7-8e5d-65becf19818b None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:05.845103 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12606c-992e-4df7-8e5d-65becf19818b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:05.845103 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12606c-992e-4df7-8e5d-65becf19818b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:05.848752 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 377/782] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Jul 20 15:13:05 2022] GET /identity/v3/services => generated 772 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:05.972922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f42db-f1ee-49d1-bfd6-4b384b03475e None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:06.024187 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf2744-e34a-4437-8aad-3f270537ca38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:06.025402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf2744-e34a-4437-8aad-3f270537ca38 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:06.025904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf2744-e34a-4437-8aad-3f270537ca38 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:06.026433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf2744-e34a-4437-8aad-3f270537ca38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:06.038028 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf2744-e34a-4437-8aad-3f270537ca38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:13:06.046054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-81cf2744-e34a-4437-8aad-3f270537ca38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:13:06.046054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:06.046054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:06.046054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:06.046054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:06.046054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:06.046054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:06.046054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:06.046054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:06.046054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:06.046054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:06.046054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:06.046054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:13:06.046054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:13:06.046054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:13:06.046054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:13:06.047185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:13:06.047185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:13:06.047185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:13:06.047185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:13:06.047185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:06.047185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:06.047185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:06.047185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:06.047185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:06.047185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:06.047185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:06.047185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:06.047185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:06.047185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:06.047185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:06.047185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:06.048060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:06.048060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:06.048060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:06.048060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:06.048060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:13:06.048060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:13:06.048060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:13:06.048060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:13:06.048060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:13:06.048060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:13:06.048060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:13:06.048060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:06.050962 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 406/783] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:13:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:06.059135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12606c-992e-4df7-8e5d-65becf19818b None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:06.166159 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12606c-992e-4df7-8e5d-65becf19818b None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:06.192471 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38c4d4-d9d5-4cfb-9bed-2bcca0a82f78 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:06.192810 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38c4d4-d9d5-4cfb-9bed-2bcca0a82f78 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:06.193037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38c4d4-d9d5-4cfb-9bed-2bcca0a82f78 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:06.194699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 407/784] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:06.199625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c5747-0059-4759-beb1-b8f7690f736a None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:06.199913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c5747-0059-4759-beb1-b8f7690f736a None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:06.200194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c5747-0059-4759-beb1-b8f7690f736a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:06.202022 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd94b2e-6432-405f-82f7-6c0311c655ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:06.202768 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd94b2e-6432-405f-82f7-6c0311c655ec None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:06.203056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd94b2e-6432-405f-82f7-6c0311c655ec None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:06.203330 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd94b2e-6432-405f-82f7-6c0311c655ec None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:06.204074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd94b2e-6432-405f-82f7-6c0311c655ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:06.204377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd94b2e-6432-405f-82f7-6c0311c655ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:06.205735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd94b2e-6432-405f-82f7-6c0311c655ec None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:06.214582 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 378/785] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:13:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:06.223156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd94b2e-6432-405f-82f7-6c0311c655ec None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:06.243328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-4a0c5747-0059-4759-beb1-b8f7690f736a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:06.244546 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-4a0c5747-0059-4759-beb1-b8f7690f736a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:06.254663 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:06.330568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd94b2e-6432-405f-82f7-6c0311c655ec None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:06.368147 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f13bc-3e0d-4a44-9c77-bf81ce385b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:06.368870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f13bc-3e0d-4a44-9c77-bf81ce385b96 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:06.369212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f13bc-3e0d-4a44-9c77-bf81ce385b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:06.369493 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f13bc-3e0d-4a44-9c77-bf81ce385b96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:06.381316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f13bc-3e0d-4a44-9c77-bf81ce385b96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:06.382857 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f13bc-3e0d-4a44-9c77-bf81ce385b96 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:06.401485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-551f13bc-3e0d-4a44-9c77-bf81ce385b96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:06.401485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:06.401485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:06.401485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:06.401485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:06.401485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:06.401485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:06.401485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:06.401485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:06.401485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:06.401485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:06.401485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:06.401485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:06.401485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:06.401485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:06.402597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:06.402597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:06.402597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:06.402597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:06.402597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:06.402597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:06.402597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:06.402597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:06.402597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:06.402597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:06.402597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:06.402597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:06.402597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:06.402597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:06.402597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:06.402597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:06.402597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:06.404089 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:06.404089 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:06.404089 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:06.404089 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:06.404089 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 379/786] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:06.408512 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f13bc-3e0d-4a44-9c77-bf81ce385b96 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:06.439980 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 408/787] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:06.472552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:06.475310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c5747-0059-4759-beb1-b8f7690f736a None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:06.498330 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:06.513416 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f13bc-3e0d-4a44-9c77-bf81ce385b96 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:06.548562 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174b4bf-c7b0-4156-b8f4-155abf9b2c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:06.549236 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174b4bf-c7b0-4156-b8f4-155abf9b2c04 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:06.549526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174b4bf-c7b0-4156-b8f4-155abf9b2c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:06.550383 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174b4bf-c7b0-4156-b8f4-155abf9b2c04 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:06.551172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174b4bf-c7b0-4156-b8f4-155abf9b2c04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:06.551490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174b4bf-c7b0-4156-b8f4-155abf9b2c04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:06.552902 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174b4bf-c7b0-4156-b8f4-155abf9b2c04 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:06.566328 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 380/788] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:06.571690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174b4bf-c7b0-4156-b8f4-155abf9b2c04 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:06.667398 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c5747-0059-4759-beb1-b8f7690f736a None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:06.675557 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174b4bf-c7b0-4156-b8f4-155abf9b2c04 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:06.700986 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:06.712206 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1dd2f1-d846-4899-8955-4da0447981f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:06.713033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1dd2f1-d846-4899-8955-4da0447981f1 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:06.713316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1dd2f1-d846-4899-8955-4da0447981f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:06.720420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1dd2f1-d846-4899-8955-4da0447981f1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:06.723132 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1dd2f1-d846-4899-8955-4da0447981f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:06.724814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1dd2f1-d846-4899-8955-4da0447981f1 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:06.742608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-3c1dd2f1-d846-4899-8955-4da0447981f1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:13:06.742608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:06.742608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:06.742608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:06.742608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:06.742608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:06.742608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:06.742608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:06.742608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:06.742608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:06.742608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:06.742608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:06.742608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:13:06.742608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:13:06.742608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:13:06.742608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:06.743644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:06.745175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:06.745175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:06.745175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:13:06.745175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:13:06.745175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:13:06.745175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:13:06.745175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:13:06.745175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:13:06.745175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:13:06.745175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:06.745877 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77196ea9-39f7-4f61-938c-a0bcf5b145cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:06.745968 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 381/789] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Jul 20 15:13:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:06.746037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77196ea9-39f7-4f61-938c-a0bcf5b145cb None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:06.746037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77196ea9-39f7-4f61-938c-a0bcf5b145cb None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:06.746037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77196ea9-39f7-4f61-938c-a0bcf5b145cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:06.750714 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1dd2f1-d846-4899-8955-4da0447981f1 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:06.767845 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77196ea9-39f7-4f61-938c-a0bcf5b145cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:13:06.784506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-77196ea9-39f7-4f61-938c-a0bcf5b145cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:13:06.784506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:06.784506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:06.784506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:06.784506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:06.784506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:06.784506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:06.784506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:06.784506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:06.784506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:06.784506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:06.784506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:06.784506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:13:06.784506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:13:06.784506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:13:06.784506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:13:06.786662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:13:06.786662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:13:06.786662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:13:06.786662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:13:06.786662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:06.786662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:06.786662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:06.786662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:06.786662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:06.786662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:06.786662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:06.786662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:06.786662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:06.786662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:06.786662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:06.786662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:06.788692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:06.788692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:06.788692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:06.788692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:06.788692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:13:06.788692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:13:06.788692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:13:06.788692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:13:06.788692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:13:06.788692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:13:06.788692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:13:06.788692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:06.794283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 409/790] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:13:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:06.795208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77196ea9-39f7-4f61-938c-a0bcf5b145cb None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:06.907912 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1dd2f1-d846-4899-8955-4da0447981f1 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:06.914603 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:06.979936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77196ea9-39f7-4f61-938c-a0bcf5b145cb None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:06.984659 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec0274-73b3-4710-94cf-527579928b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:06.985657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec0274-73b3-4710-94cf-527579928b31 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:06.985836 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec0274-73b3-4710-94cf-527579928b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:06.985988 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec0274-73b3-4710-94cf-527579928b31 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:06.986837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec0274-73b3-4710-94cf-527579928b31 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:06.987774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec0274-73b3-4710-94cf-527579928b31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:06.989297 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec0274-73b3-4710-94cf-527579928b31 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:07.008363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 382/791] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Jul 20 15:13:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:07.013834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec0274-73b3-4710-94cf-527579928b31 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:07.060289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:07.066949 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc0e30-30ab-4300-b9e6-1c7e9b452e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:07.074871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc0e30-30ab-4300-b9e6-1c7e9b452e62 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:07.074871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc0e30-30ab-4300-b9e6-1c7e9b452e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:07.074871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc0e30-30ab-4300-b9e6-1c7e9b452e62 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:07.074871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc0e30-30ab-4300-b9e6-1c7e9b452e62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:07.074871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc0e30-30ab-4300-b9e6-1c7e9b452e62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:07.074871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc0e30-30ab-4300-b9e6-1c7e9b452e62 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:07.107927 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 410/792] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:13:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:07.112918 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc0e30-30ab-4300-b9e6-1c7e9b452e62 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:07.127401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec0274-73b3-4710-94cf-527579928b31 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:07.165087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44866216-7359-4c34-9868-2825580ab035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:07.166277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44866216-7359-4c34-9868-2825580ab035 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:07.166766 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44866216-7359-4c34-9868-2825580ab035 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:07.167241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44866216-7359-4c34-9868-2825580ab035 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:07.180796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44866216-7359-4c34-9868-2825580ab035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:07.183233 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44866216-7359-4c34-9868-2825580ab035 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:07.203524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-44866216-7359-4c34-9868-2825580ab035 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:07.203524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:07.203524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:07.203524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:07.203524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:07.203524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:07.203524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:07.203524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:07.203524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:07.203524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:07.203524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:07.203524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:07.203524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:07.203524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:07.203524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:07.204490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:07.204490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:07.204490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:07.204490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:07.204490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:07.204490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:07.204490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:07.204490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:07.204490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:07.204490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:07.204490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:07.204490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:07.204490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:07.204490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:07.204490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:07.204490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:07.204490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:07.205404 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:07.205404 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:07.205404 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:07.205404 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:07.208343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 383/793] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:07.214069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44866216-7359-4c34-9868-2825580ab035 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:07.315744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:07.323639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44866216-7359-4c34-9868-2825580ab035 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:07.360941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d375cc6-d14f-45e2-bf7a-9939bc6d4891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:07.362472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d375cc6-d14f-45e2-bf7a-9939bc6d4891 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:07.362958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d375cc6-d14f-45e2-bf7a-9939bc6d4891 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:07.363447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d375cc6-d14f-45e2-bf7a-9939bc6d4891 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:07.364365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d375cc6-d14f-45e2-bf7a-9939bc6d4891 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:07.364869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d375cc6-d14f-45e2-bf7a-9939bc6d4891 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:07.366507 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d375cc6-d14f-45e2-bf7a-9939bc6d4891 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:07.372569 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc0e30-30ab-4300-b9e6-1c7e9b452e62 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:07.380973 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 384/794] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:07.386724 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d375cc6-d14f-45e2-bf7a-9939bc6d4891 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:07.445484 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:07.466970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725e5e5-8dbd-4bfb-8423-3407c65f460e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:07.467790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725e5e5-8dbd-4bfb-8423-3407c65f460e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:07.468079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725e5e5-8dbd-4bfb-8423-3407c65f460e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:07.468367 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725e5e5-8dbd-4bfb-8423-3407c65f460e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:07.495373 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725e5e5-8dbd-4bfb-8423-3407c65f460e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:07.496857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725e5e5-8dbd-4bfb-8423-3407c65f460e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:07.500595 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d375cc6-d14f-45e2-bf7a-9939bc6d4891 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:07.529035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-e725e5e5-8dbd-4bfb-8423-3407c65f460e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:07.529035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:07.529035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:07.529035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:07.529035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:07.529035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:07.529035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:07.529035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:07.529035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:07.529035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:07.529035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:07.529035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:07.529035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:07.529035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:07.529035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:07.530007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:07.530007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:07.530007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:07.530007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:07.530007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:07.530007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:07.530007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:07.530007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:07.530007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:07.530007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:07.530007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:07.530007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:07.530007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:07.530007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:07.530007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:07.530007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:07.530007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:07.530959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:07.530959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:07.530959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:07.530959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:07.534911 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 411/795] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:07.536417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc31ea1-8114-40ff-8610-28be6a55c39c None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:07.536714 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc31ea1-8114-40ff-8610-28be6a55c39c None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:07.537000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc31ea1-8114-40ff-8610-28be6a55c39c None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:07.538526 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 412/796] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:07.539234 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc31ea1-8114-40ff-8610-28be6a55c39c None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:07.542078 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3638f39-6eda-4fa1-931c-30d3c5d6670a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:07.543142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3638f39-6eda-4fa1-931c-30d3c5d6670a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:07.543714 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3638f39-6eda-4fa1-931c-30d3c5d6670a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:07.544198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3638f39-6eda-4fa1-931c-30d3c5d6670a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:07.551752 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3638f39-6eda-4fa1-931c-30d3c5d6670a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:07.553571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3638f39-6eda-4fa1-931c-30d3c5d6670a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:07.569773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-e3638f39-6eda-4fa1-931c-30d3c5d6670a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:07.569773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:07.569773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:07.569773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:07.569773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:07.569773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:07.569773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:07.569773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:07.569773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:07.569773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:07.569773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:07.569773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:07.569773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:13:07.569773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:13:07.569773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:13:07.570786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:13:07.570786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:07.570786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:07.570786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:07.570786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:07.570786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:07.570786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:07.570786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:07.570786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:07.570786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:07.570786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:07.570786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:07.570786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:07.570786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:07.570786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:07.570786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:07.570786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:07.571708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:13:07.571708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:13:07.571708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:13:07.571708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:13:07.571708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:13:07.571708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:13:07.571708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:07.571708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:07.574253 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 385/797] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:07.579626 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b81333-6b36-470e-8eea-8eadc894f045 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:07.579918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b81333-6b36-470e-8eea-8eadc894f045 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:07.580195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b81333-6b36-470e-8eea-8eadc894f045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:07.604568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-f8b81333-6b36-470e-8eea-8eadc894f045 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:07.606149 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f8b81333-6b36-470e-8eea-8eadc894f045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:07.616723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:07.678073 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc31ea1-8114-40ff-8610-28be6a55c39c None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:07.680415 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 386/798] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:07.681647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b81333-6b36-470e-8eea-8eadc894f045 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:07.737702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100e897-56df-4370-80e8-a8e3f8708165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:07.738438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100e897-56df-4370-80e8-a8e3f8708165 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:07.738726 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100e897-56df-4370-80e8-a8e3f8708165 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:07.739005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100e897-56df-4370-80e8-a8e3f8708165 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:07.739740 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100e897-56df-4370-80e8-a8e3f8708165 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:07.740048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100e897-56df-4370-80e8-a8e3f8708165 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:07.741430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100e897-56df-4370-80e8-a8e3f8708165 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:07.764738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 413/799] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:07.766003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100e897-56df-4370-80e8-a8e3f8708165 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:07.820060 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b81333-6b36-470e-8eea-8eadc894f045 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:07.821108 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:07.866770 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa7c73-9937-49f9-b2bc-3bcb482dbd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:07.867445 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa7c73-9937-49f9-b2bc-3bcb482dbd3c None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:07.868236 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa7c73-9937-49f9-b2bc-3bcb482dbd3c None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:07.868535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa7c73-9937-49f9-b2bc-3bcb482dbd3c None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:07.869622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa7c73-9937-49f9-b2bc-3bcb482dbd3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:07.869939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa7c73-9937-49f9-b2bc-3bcb482dbd3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:07.872244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa7c73-9937-49f9-b2bc-3bcb482dbd3c None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:07.880426 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100e897-56df-4370-80e8-a8e3f8708165 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:07.891957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 387/800] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:13:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:07.891957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa7c73-9937-49f9-b2bc-3bcb482dbd3c None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:07.897028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:07.915841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb810a-ec8c-4068-baee-481de31a048a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:07.916635 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb810a-ec8c-4068-baee-481de31a048a None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:07.917006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb810a-ec8c-4068-baee-481de31a048a None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:07.917278 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb810a-ec8c-4068-baee-481de31a048a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:07.928209 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb810a-ec8c-4068-baee-481de31a048a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:13:07.935926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-bafb810a-ec8c-4068-baee-481de31a048a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:13:07.935926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:07.935926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:07.935926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:07.935926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:07.935926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:07.935926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:07.935926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:07.935926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:07.935926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:07.935926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:07.935926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:07.935926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:13:07.935926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:13:07.935926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:13:07.935926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:13:07.936875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:13:07.936875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:13:07.936875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:13:07.936875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:13:07.936875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:07.936875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:07.936875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:07.936875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:07.936875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:07.936875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:07.936875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:07.936875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:07.936875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:07.936875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:07.936875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:07.936875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:07.937739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:07.937739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:07.937739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:07.937739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:07.937739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:13:07.937739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:13:07.937739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:13:07.937739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:13:07.937739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:13:07.937739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:13:07.937739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:13:07.937739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:07.937739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 414/801] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:13:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:07.939029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ec3b0-fda6-4019-b93e-1b65d2cbdc6d None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:07.939029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ec3b0-fda6-4019-b93e-1b65d2cbdc6d None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:07.939029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ec3b0-fda6-4019-b93e-1b65d2cbdc6d None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:07.942304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 415/802] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:07.942304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ec3b0-fda6-4019-b93e-1b65d2cbdc6d None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:08.010236 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa7c73-9937-49f9-b2bc-3bcb482dbd3c None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:08.046114 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac8f94-16f6-4e65-bfa1-fe1e9e4a4582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:08.046936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac8f94-16f6-4e65-bfa1-fe1e9e4a4582 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:08.047219 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac8f94-16f6-4e65-bfa1-fe1e9e4a4582 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:08.047495 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac8f94-16f6-4e65-bfa1-fe1e9e4a4582 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:08.051000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ec3b0-fda6-4019-b93e-1b65d2cbdc6d None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:08.052649 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:08.055380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac8f94-16f6-4e65-bfa1-fe1e9e4a4582 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:08.056812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac8f94-16f6-4e65-bfa1-fe1e9e4a4582 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:08.071181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-d9ac8f94-16f6-4e65-bfa1-fe1e9e4a4582 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:13:08.071181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:08.071181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:08.071181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:08.071181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:08.071181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:08.071181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:08.071181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:08.071181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:08.071181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:08.071181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:08.071181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:08.071181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:13:08.071181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:13:08.071181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:13:08.071181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:08.077203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:08.078476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:08.078476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:08.078476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:13:08.078476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:13:08.078476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:13:08.078476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:13:08.078476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:13:08.078476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:13:08.078476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:13:08.078476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:08.078476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 388/803] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:13:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:08.078476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac8f94-16f6-4e65-bfa1-fe1e9e4a4582 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:08.102445 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a38ce-e2a0-4a98-98f9-4777933d53c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:08.103351 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a38ce-e2a0-4a98-98f9-4777933d53c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:08.103632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a38ce-e2a0-4a98-98f9-4777933d53c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:08.103907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a38ce-e2a0-4a98-98f9-4777933d53c2 None admin] PATH_INFO: `/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/4e75a82cc7e94702ba0080fc65d6eef3/roles/d820b1d3fa4144aca575c04645c099bc` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:08.118486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a38ce-e2a0-4a98-98f9-4777933d53c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdae1cb71dd449b8a18062b7b52889e4, user_id=4e75a82cc7e94702ba0080fc65d6eef3, role_id=d820b1d3fa4144aca575c04645c099bc)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:08.119922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a38ce-e2a0-4a98-98f9-4777933d53c2 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:08.149421 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:08.185014 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac8f94-16f6-4e65-bfa1-fe1e9e4a4582 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:08.217572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d42506-c4b5-49d3-9d86-21c9da41b0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:08.218238 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d42506-c4b5-49d3-9d86-21c9da41b0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:08.218554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d42506-c4b5-49d3-9d86-21c9da41b0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:08.218830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d42506-c4b5-49d3-9d86-21c9da41b0a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:08.219568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d42506-c4b5-49d3-9d86-21c9da41b0a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:08.219874 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d42506-c4b5-49d3-9d86-21c9da41b0a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:08.221229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d42506-c4b5-49d3-9d86-21c9da41b0a0 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:08.229144 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 389/804] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:13:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:08.230411 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3064b4-665c-446d-a3f2-92258822ba32 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:08.230701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3064b4-665c-446d-a3f2-92258822ba32 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:08.230979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3064b4-665c-446d-a3f2-92258822ba32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:08.246793 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-8e3064b4-665c-446d-a3f2-92258822ba32 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:08.247973 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8e3064b4-665c-446d-a3f2-92258822ba32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:08.256738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 416/805] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:13:07 2022] PUT /identity/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/4e75a82cc7e94702ba0080fc65d6eef3/roles/d820b1d3fa4144aca575c04645c099bc => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:13:08.260167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a38ce-e2a0-4a98-98f9-4777933d53c2 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:08.326058 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 390/806] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:08.327021 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3064b4-665c-446d-a3f2-92258822ba32 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:08.380164 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a38ce-e2a0-4a98-98f9-4777933d53c2 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:08.404183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:08.424453 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8143e2-fc04-4ef1-afcb-a80e41046e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:08.425181 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8143e2-fc04-4ef1-afcb-a80e41046e0e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:08.425470 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8143e2-fc04-4ef1-afcb-a80e41046e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:08.425745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8143e2-fc04-4ef1-afcb-a80e41046e0e None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:08.426615 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8143e2-fc04-4ef1-afcb-a80e41046e0e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:08.426930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8143e2-fc04-4ef1-afcb-a80e41046e0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:08.428371 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8143e2-fc04-4ef1-afcb-a80e41046e0e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:08.446634 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 417/807] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Jul 20 15:13:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:08.450311 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278d63c-08eb-449b-972d-c3d8342a88c0 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:08.450311 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278d63c-08eb-449b-972d-c3d8342a88c0 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:08.450311 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278d63c-08eb-449b-972d-c3d8342a88c0 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:08.450311 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 418/808] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:08.450605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278d63c-08eb-449b-972d-c3d8342a88c0 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:08.463418 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:08.464665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3064b4-665c-446d-a3f2-92258822ba32 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:08.507276 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da718e90-001a-47e4-8852-cef653033c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:08.508898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da718e90-001a-47e4-8852-cef653033c43 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:08.509682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da718e90-001a-47e4-8852-cef653033c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:08.510134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da718e90-001a-47e4-8852-cef653033c43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:08.525249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da718e90-001a-47e4-8852-cef653033c43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:08.527414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da718e90-001a-47e4-8852-cef653033c43 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:08.551038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-da718e90-001a-47e4-8852-cef653033c43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:08.551038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:08.551038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:08.551038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:08.551038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:08.551038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:08.551038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:08.551038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:08.551038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:08.551038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:08.551038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:08.551038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:08.551038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:08.551038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:08.551038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:08.552068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:08.552068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:08.552068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:08.552068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:08.552068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:08.552068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:08.552068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:08.552068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:08.552068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:08.552068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:08.552068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:08.552068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:08.552068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:08.552068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:08.552068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:08.552068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:08.552068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:08.557825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:08.557825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:08.557825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:08.557825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:08.557825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 391/809] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:08.557825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da718e90-001a-47e4-8852-cef653033c43 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:08.558444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278d63c-08eb-449b-972d-c3d8342a88c0 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:08.594834 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a36a34-95e4-4b09-b4f0-542c063d9891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:08.595558 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a36a34-95e4-4b09-b4f0-542c063d9891 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:08.595840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a36a34-95e4-4b09-b4f0-542c063d9891 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:08.596119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a36a34-95e4-4b09-b4f0-542c063d9891 None admin] PATH_INFO: `/v3/services/network` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:08.596799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a36a34-95e4-4b09-b4f0-542c063d9891 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:08.598175 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a36a34-95e4-4b09-b4f0-542c063d9891 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:08.610111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-43a36a34-95e4-4b09-b4f0-542c063d9891 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:13:08.610111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:08.610111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:08.610111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:08.610111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:08.610111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:08.610111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:08.610111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:08.610111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:13:08.610111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:08.610111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:08.610111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:08.610111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:08.610111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:08.610111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:08.610111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:08.610111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:08.611457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:08.611457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:08.611457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:08.611457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:08.611457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:08.611457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:08.611457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:08.611457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:08.611457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:08.611457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:08.611457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:08.611457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:08.611457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:08.611457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:08.611457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:08.611457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:08.613076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:08.613076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:08.613076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:08.613076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:08.613076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:08.613076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:08.613076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:08.613076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:08.613076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:08.613076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:08.613076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:08.613076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:08.613076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 15:13:08.613076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:08.616539 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 419/810] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:08 2022] GET /identity/v3/services/network => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:08.617882 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa09323-b737-4c37-b8f3-c9c43640dca4 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:08.618174 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa09323-b737-4c37-b8f3-c9c43640dca4 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:08.618490 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa09323-b737-4c37-b8f3-c9c43640dca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:08.635511 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-efa09323-b737-4c37-b8f3-c9c43640dca4 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:08.636888 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-efa09323-b737-4c37-b8f3-c9c43640dca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:08.665945 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da718e90-001a-47e4-8852-cef653033c43 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:08.676002 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:08.702573 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da02c3-395b-488e-99ac-ae07d75ea1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:08.703378 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da02c3-395b-488e-99ac-ae07d75ea1ac None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:08.703658 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da02c3-395b-488e-99ac-ae07d75ea1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:08.703926 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da02c3-395b-488e-99ac-ae07d75ea1ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:08.710604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 420/811] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:08.710604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa09323-b737-4c37-b8f3-c9c43640dca4 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:08.715731 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da02c3-395b-488e-99ac-ae07d75ea1ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:08.717169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da02c3-395b-488e-99ac-ae07d75ea1ac None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:08.717348 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:08.735244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-15da02c3-395b-488e-99ac-ae07d75ea1ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:08.735244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:08.735244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:08.735244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:08.735244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:08.735244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:08.735244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:08.735244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:08.735244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:08.735244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:08.735244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:08.735244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:08.735244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:08.735244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:08.735244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:08.736208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:08.736208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:08.736208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:08.736208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:08.736208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:08.736208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:08.736208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:08.736208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:08.736208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:08.736208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:08.736208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:08.736208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:08.736208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:08.736208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:08.736208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:08.736208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:08.736208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:08.737741 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:08.737741 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:08.737741 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:08.737741 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:08.737741 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 392/812] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:08.737741 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da02c3-395b-488e-99ac-ae07d75ea1ac None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:08.830747 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa09323-b737-4c37-b8f3-c9c43640dca4 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:08.843511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da02c3-395b-488e-99ac-ae07d75ea1ac None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:08.867768 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8be140-dc82-4a64-ae00-3f1aac0c02b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:08.868427 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8be140-dc82-4a64-ae00-3f1aac0c02b6 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:08.868645 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8be140-dc82-4a64-ae00-3f1aac0c02b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:08.868923 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8be140-dc82-4a64-ae00-3f1aac0c02b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:08.869650 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8be140-dc82-4a64-ae00-3f1aac0c02b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:08.869960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8be140-dc82-4a64-ae00-3f1aac0c02b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:08.871376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8be140-dc82-4a64-ae00-3f1aac0c02b6 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:08.878653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55644c9-c9c7-4b14-b0c9-9d805c2c0d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:08.879318 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55644c9-c9c7-4b14-b0c9-9d805c2c0d93 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:08.879606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55644c9-c9c7-4b14-b0c9-9d805c2c0d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:08.879876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55644c9-c9c7-4b14-b0c9-9d805c2c0d93 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:08.880585 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55644c9-c9c7-4b14-b0c9-9d805c2c0d93 None admin] RBAC: Adding query filter params (name=network) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:08.880886 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55644c9-c9c7-4b14-b0c9-9d805c2c0d93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:08.882232 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55644c9-c9c7-4b14-b0c9-9d805c2c0d93 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:08.888721 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 421/813] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:08.888721 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8be140-dc82-4a64-ae00-3f1aac0c02b6 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:08.891249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 393/814] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Jul 20 15:13:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:08.896381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55644c9-c9c7-4b14-b0c9-9d805c2c0d93 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:08.918674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:08.959897 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:08.994135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55644c9-c9c7-4b14-b0c9-9d805c2c0d93 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:09.000294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8be140-dc82-4a64-ae00-3f1aac0c02b6 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:09.031712 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58027919-e9a1-4d3f-9f9d-c5e19c1a40f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:09.032375 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58027919-e9a1-4d3f-9f9d-c5e19c1a40f1 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:09.032665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58027919-e9a1-4d3f-9f9d-c5e19c1a40f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:09.032956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58027919-e9a1-4d3f-9f9d-c5e19c1a40f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:09.033680 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58027919-e9a1-4d3f-9f9d-c5e19c1a40f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:09.034018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58027919-e9a1-4d3f-9f9d-c5e19c1a40f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:09.035421 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58027919-e9a1-4d3f-9f9d-c5e19c1a40f1 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:09.037354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87f9dc-70eb-481a-a23a-2302d8f5bef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:09.038611 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87f9dc-70eb-481a-a23a-2302d8f5bef4 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:09.038897 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87f9dc-70eb-481a-a23a-2302d8f5bef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:09.039173 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87f9dc-70eb-481a-a23a-2302d8f5bef4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:09.039825 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87f9dc-70eb-481a-a23a-2302d8f5bef4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:09.041182 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87f9dc-70eb-481a-a23a-2302d8f5bef4 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:09.051580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 394/815] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:09.051580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58027919-e9a1-4d3f-9f9d-c5e19c1a40f1 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:09.053684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-2c87f9dc-70eb-481a-a23a-2302d8f5bef4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:13:09.053684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:09.053684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:09.053684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:09.053684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:09.053684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:09.053684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:09.053684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:09.053684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:13:09.053684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:09.053684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:09.053684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:09.053684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:09.053684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:09.053684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:09.053684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:09.054670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:09.054670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:09.054670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:09.054670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:09.054670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:09.054670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:09.054670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:09.054670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:09.054670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:09.054670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:09.054670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:09.054670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:09.054670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:09.054670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:09.054670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:09.054670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:09.055528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:09.055528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:09.055528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:09.055528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:09.055528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:09.055528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:09.055528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:09.055528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:09.055528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:09.055528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:09.055528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:09.055528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:09.055528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:09.055528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 15:13:09.055528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:09.055528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 422/816] 10.222.0.78 () {56 vars in 1129 bytes} [Wed Jul 20 15:13:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:09.056381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87f9dc-70eb-481a-a23a-2302d8f5bef4 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:09.148729 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:09.164692 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58027919-e9a1-4d3f-9f9d-c5e19c1a40f1 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:09.171276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87f9dc-70eb-481a-a23a-2302d8f5bef4 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:09.199121 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:09.201078 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a4281-95ee-4a58-aef6-29584fb48340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:09.201873 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a4281-95ee-4a58-aef6-29584fb48340 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:09.202161 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a4281-95ee-4a58-aef6-29584fb48340 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:09.202494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a4281-95ee-4a58-aef6-29584fb48340 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:09.207902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d888b8-cf34-4121-9a3d-b441f2696089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:09.208824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d888b8-cf34-4121-9a3d-b441f2696089 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:09.209306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d888b8-cf34-4121-9a3d-b441f2696089 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:09.209762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d888b8-cf34-4121-9a3d-b441f2696089 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:09.210588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a4281-95ee-4a58-aef6-29584fb48340 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:09.210842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d888b8-cf34-4121-9a3d-b441f2696089 None admin] RBAC: Adding query filter params (type=network) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:09.211316 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d888b8-cf34-4121-9a3d-b441f2696089 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:09.212048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a4281-95ee-4a58-aef6-29584fb48340 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:09.213222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d888b8-cf34-4121-9a3d-b441f2696089 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:09.218656 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 423/817] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Jul 20 15:13:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:09.219677 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d888b8-cf34-4121-9a3d-b441f2696089 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:09.226423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-011a4281-95ee-4a58-aef6-29584fb48340 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:13:09.226423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:09.226423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:09.226423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:09.226423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:09.226423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:09.226423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:09.226423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:09.226423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:09.226423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:09.226423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:09.226423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:09.226423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:13:09.226423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:13:09.226423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:13:09.226423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:09.227385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:09.228354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:09.228354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:09.228354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:13:09.228354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:13:09.228354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:13:09.228354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:13:09.228354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:13:09.228354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:13:09.228354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:13:09.228354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:09.228354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 395/818] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Jul 20 15:13:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:09.229028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a4281-95ee-4a58-aef6-29584fb48340 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:09.331790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d888b8-cf34-4121-9a3d-b441f2696089 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:09.335765 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a4281-95ee-4a58-aef6-29584fb48340 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:09.368391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf05058-638c-4887-acde-a89c22760e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:09.369211 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf05058-638c-4887-acde-a89c22760e5c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:09.369491 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf05058-638c-4887-acde-a89c22760e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:09.369776 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf05058-638c-4887-acde-a89c22760e5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:09.371158 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd84dd-c48a-403b-a280-1f37eaaab325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:09.371852 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd84dd-c48a-403b-a280-1f37eaaab325 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:09.372142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd84dd-c48a-403b-a280-1f37eaaab325 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:09.372421 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd84dd-c48a-403b-a280-1f37eaaab325 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:09.373165 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd84dd-c48a-403b-a280-1f37eaaab325 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:09.373489 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd84dd-c48a-403b-a280-1f37eaaab325 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:09.374924 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd84dd-c48a-403b-a280-1f37eaaab325 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:09.378744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf05058-638c-4887-acde-a89c22760e5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:09.380251 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 396/819] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Jul 20 15:13:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:09.380581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf05058-638c-4887-acde-a89c22760e5c None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:09.381030 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd84dd-c48a-403b-a280-1f37eaaab325 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:09.395588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:09.401765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-9cf05058-638c-4887-acde-a89c22760e5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:09.401765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:09.401765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:09.401765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:09.401765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:09.401765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:09.401765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:09.401765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:09.401765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:09.401765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:09.401765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:09.401765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:09.401765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:13:09.401765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:13:09.401765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:13:09.402731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:13:09.402731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:09.402731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:09.402731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:09.402731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:09.402731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:09.402731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:09.402731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:09.402731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:09.402731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:09.402731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:09.402731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:09.402731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:09.402731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:09.402731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:09.402731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:09.402731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:09.403673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:13:09.403673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:13:09.403673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:13:09.403673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:13:09.403673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:13:09.403673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:13:09.403673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:09.403673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:09.404396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 424/820] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:09.405002 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf05058-638c-4887-acde-a89c22760e5c None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:09.459773 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:09.493763 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd84dd-c48a-403b-a280-1f37eaaab325 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:09.513728 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf05058-638c-4887-acde-a89c22760e5c None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:09.531649 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16367c-d097-4863-8ca7-19ddfe044590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:09.532301 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16367c-d097-4863-8ca7-19ddfe044590 None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:09.532583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16367c-d097-4863-8ca7-19ddfe044590 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:09.532857 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16367c-d097-4863-8ca7-19ddfe044590 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:09.533589 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16367c-d097-4863-8ca7-19ddfe044590 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:09.535884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16367c-d097-4863-8ca7-19ddfe044590 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:09.550305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bed54-714d-4ab8-9646-e9d26303d30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:09.551073 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bed54-714d-4ab8-9646-e9d26303d30a None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:09.551423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bed54-714d-4ab8-9646-e9d26303d30a None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:09.551763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bed54-714d-4ab8-9646-e9d26303d30a None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:09.552664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471bed54-714d-4ab8-9646-e9d26303d30a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:09.553036 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471bed54-714d-4ab8-9646-e9d26303d30a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:09.554503 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471bed54-714d-4ab8-9646-e9d26303d30a None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:09.571858 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 425/821] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Jul 20 15:13:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:09.572657 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bed54-714d-4ab8-9646-e9d26303d30a None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:09.637227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 397/822] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Jul 20 15:13:09 2022] POST /identity/v3/endpoints => generated 327 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:13:09.638118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16367c-d097-4863-8ca7-19ddfe044590 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:09.682566 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bed54-714d-4ab8-9646-e9d26303d30a None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:09.696467 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:09.720435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:09.734749 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef92205-e8c7-4a60-8b32-1ddf91797b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:09.735871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef92205-e8c7-4a60-8b32-1ddf91797b16 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:09.736321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef92205-e8c7-4a60-8b32-1ddf91797b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:09.736761 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef92205-e8c7-4a60-8b32-1ddf91797b16 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:09.737804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef92205-e8c7-4a60-8b32-1ddf91797b16 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:09.738363 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef92205-e8c7-4a60-8b32-1ddf91797b16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:09.740229 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef92205-e8c7-4a60-8b32-1ddf91797b16 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:09.746999 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 426/823] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Jul 20 15:13:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:09.748305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef92205-e8c7-4a60-8b32-1ddf91797b16 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:09.767948 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16367c-d097-4863-8ca7-19ddfe044590 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:09.820551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1d90c-c902-4e43-b192-4086f5bf0f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:09.821225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1d90c-c902-4e43-b192-4086f5bf0f21 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:09.821509 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1d90c-c902-4e43-b192-4086f5bf0f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:09.821789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1d90c-c902-4e43-b192-4086f5bf0f21 None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:09.822624 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1d90c-c902-4e43-b192-4086f5bf0f21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:09.822940 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1d90c-c902-4e43-b192-4086f5bf0f21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:09.824377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1d90c-c902-4e43-b192-4086f5bf0f21 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:09.838622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 398/824] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:13:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:09.839431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1d90c-c902-4e43-b192-4086f5bf0f21 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:09.874566 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef92205-e8c7-4a60-8b32-1ddf91797b16 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:09.916804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcd507-c587-4135-a809-295937f69491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:09.917734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcd507-c587-4135-a809-295937f69491 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:09.918180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcd507-c587-4135-a809-295937f69491 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:09.918658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcd507-c587-4135-a809-295937f69491 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:09.962528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:09.967270 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1d90c-c902-4e43-b192-4086f5bf0f21 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:09.993276 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:09.994873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fcd507-c587-4135-a809-295937f69491 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:09.996343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fcd507-c587-4135-a809-295937f69491 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:10.005279 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321fefa-7025-49fd-be5d-c34d171e727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:10.005939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321fefa-7025-49fd-be5d-c34d171e727d None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:10.006227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321fefa-7025-49fd-be5d-c34d171e727d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:10.006526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321fefa-7025-49fd-be5d-c34d171e727d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:10.007267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321fefa-7025-49fd-be5d-c34d171e727d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:10.008678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321fefa-7025-49fd-be5d-c34d171e727d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:10.098880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 427/825] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:13:09 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:10.099746 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcd507-c587-4135-a809-295937f69491 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:10.134914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 399/826] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Jul 20 15:13:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:13:10.136124 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321fefa-7025-49fd-be5d-c34d171e727d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:10.185972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:10.223660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcd507-c587-4135-a809-295937f69491 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:10.276075 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a496f-cc04-4258-be9c-2a28cefdf758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:10.277440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a496f-cc04-4258-be9c-2a28cefdf758 None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:10.277826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a496f-cc04-4258-be9c-2a28cefdf758 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:10.278111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a496f-cc04-4258-be9c-2a28cefdf758 None admin] PATH_INFO: `/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/ed0d719c0eb04a3b8a228c6ce70fdf98/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:10.281081 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a496f-cc04-4258-be9c-2a28cefdf758 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdae1cb71dd449b8a18062b7b52889e4, user_id=ed0d719c0eb04a3b8a228c6ce70fdf98, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:10.283179 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a496f-cc04-4258-be9c-2a28cefdf758 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:10.292225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321fefa-7025-49fd-be5d-c34d171e727d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:10.327996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ff037-210b-47b1-be7a-149e42f27a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:10.328800 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ff037-210b-47b1-be7a-149e42f27a1e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:10.329081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ff037-210b-47b1-be7a-149e42f27a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:10.329354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ff037-210b-47b1-be7a-149e42f27a1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:10.340807 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ff037-210b-47b1-be7a-149e42f27a1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:10.342240 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ff037-210b-47b1-be7a-149e42f27a1e None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:10.360351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-695ff037-210b-47b1-be7a-149e42f27a1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:10.360351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:10.360351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:10.360351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:10.360351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:10.360351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:10.360351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:10.360351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:10.360351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:10.360351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:10.360351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:10.360351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:10.360351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:10.360351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:10.360351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:10.361494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:10.361494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:10.361494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:10.361494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:10.361494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:10.361494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:10.361494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:10.361494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:10.361494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:10.361494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:10.361494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:10.361494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:10.361494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:10.361494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:10.361494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:10.361494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:10.361494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:10.362516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:10.362516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:10.362516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:10.362516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:10.362516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 400/827] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:10.363291 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f5938-2913-4e00-9b63-c31fbaf1f875 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:10.363576 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f5938-2913-4e00-9b63-c31fbaf1f875 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:10.363851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f5938-2913-4e00-9b63-c31fbaf1f875 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:10.365249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 401/828] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:10.367005 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f5938-2913-4e00-9b63-c31fbaf1f875 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:10.399832 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:10.438640 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 428/829] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:13:10 2022] PUT /identity/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/ed0d719c0eb04a3b8a228c6ce70fdf98/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:13:10.439988 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6126222-dcf7-4971-8be3-73a3aff833b7 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:10.440778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6126222-dcf7-4971-8be3-73a3aff833b7 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:10.441053 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6126222-dcf7-4971-8be3-73a3aff833b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:10.467627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-c6126222-dcf7-4971-8be3-73a3aff833b7 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:10.469069 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c6126222-dcf7-4971-8be3-73a3aff833b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:10.483500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f5938-2913-4e00-9b63-c31fbaf1f875 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:10.526294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9f48c-6e13-4648-9879-9c22eeefbd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:10.528000 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9f48c-6e13-4648-9879-9c22eeefbd3b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:10.529423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9f48c-6e13-4648-9879-9c22eeefbd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:10.529950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9f48c-6e13-4648-9879-9c22eeefbd3b None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:10.531039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9f48c-6e13-4648-9879-9c22eeefbd3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:10.531568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9f48c-6e13-4648-9879-9c22eeefbd3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:10.533200 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9f48c-6e13-4648-9879-9c22eeefbd3b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:10.554680 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 402/830] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:10.555417 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9f48c-6e13-4648-9879-9c22eeefbd3b None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:10.574291 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 429/831] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:10.598208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6126222-dcf7-4971-8be3-73a3aff833b7 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:10.667273 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:10.674703 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:10.675631 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9f48c-6e13-4648-9879-9c22eeefbd3b None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:10.711927 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac619a4e-ba0e-46d9-98a5-bb930c839863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:10.712972 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac619a4e-ba0e-46d9-98a5-bb930c839863 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:10.713401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac619a4e-ba0e-46d9-98a5-bb930c839863 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:10.713849 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac619a4e-ba0e-46d9-98a5-bb930c839863 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:10.720966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac619a4e-ba0e-46d9-98a5-bb930c839863 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:10.721711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6126222-dcf7-4971-8be3-73a3aff833b7 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:10.722963 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac619a4e-ba0e-46d9-98a5-bb930c839863 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:10.737918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-ac619a4e-ba0e-46d9-98a5-bb930c839863 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:10.737918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:10.737918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:10.737918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:10.737918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:10.737918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:10.737918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:10.737918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:10.737918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:10.737918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:10.737918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:10.737918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:10.737918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:13:10.737918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:13:10.737918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:13:10.739084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:13:10.739084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:10.739084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:10.739084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:10.739084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:10.739084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:10.739084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:10.739084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:10.739084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:10.739084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:10.739084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:10.739084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:10.739084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:10.739084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:10.739084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:10.739084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:10.739084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:10.740220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:13:10.740220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:13:10.740220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:13:10.740220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:13:10.740220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:13:10.740220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:13:10.740220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:10.740220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:10.742324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 403/832] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:10.751520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac619a4e-ba0e-46d9-98a5-bb930c839863 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:10.758235 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763ec2b-bee4-4901-900d-29f11add3517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:10.763067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763ec2b-bee4-4901-900d-29f11add3517 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:10.764006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763ec2b-bee4-4901-900d-29f11add3517 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:10.764526 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763ec2b-bee4-4901-900d-29f11add3517 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:10.778810 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5763ec2b-bee4-4901-900d-29f11add3517 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:13:10.789833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-5763ec2b-bee4-4901-900d-29f11add3517 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:13:10.789833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:10.789833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:10.789833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:10.789833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:10.789833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:10.789833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:10.789833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:10.789833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:10.789833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:10.789833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:10.789833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:10.789833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:13:10.789833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:13:10.789833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:13:10.789833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:13:10.793678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:13:10.793678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:13:10.793678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:13:10.793678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:13:10.793678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:10.793678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:10.793678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:10.793678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:10.793678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:10.793678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:10.793678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:10.793678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:10.793678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:10.793678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:10.793678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:10.793678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:10.794572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:10.794572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:10.794572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:10.794572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:10.794572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:13:10.794572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:13:10.794572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:13:10.794572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:13:10.794572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:13:10.794572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:13:10.794572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:13:10.794572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:10.794572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 430/833] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:13:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:10.799145 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763ec2b-bee4-4901-900d-29f11add3517 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:10.872267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac619a4e-ba0e-46d9-98a5-bb930c839863 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:10.920617 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ed40c-e025-4554-adac-53f467727400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:10.921310 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ed40c-e025-4554-adac-53f467727400 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:10.921596 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ed40c-e025-4554-adac-53f467727400 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:10.921874 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ed40c-e025-4554-adac-53f467727400 None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:10.922694 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ed40c-e025-4554-adac-53f467727400 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:10.923005 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ed40c-e025-4554-adac-53f467727400 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:10.924456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ed40c-e025-4554-adac-53f467727400 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:10.927288 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763ec2b-bee4-4901-900d-29f11add3517 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:10.941895 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 404/834] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:13:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:10.942831 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ed40c-e025-4554-adac-53f467727400 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:10.954990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:10.962992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:10.974717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dadb81-4418-4ced-9768-62aedbfe18a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:10.976171 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dadb81-4418-4ced-9768-62aedbfe18a6 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:10.976638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dadb81-4418-4ced-9768-62aedbfe18a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:10.977064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dadb81-4418-4ced-9768-62aedbfe18a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:10.977953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dadb81-4418-4ced-9768-62aedbfe18a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:10.978605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dadb81-4418-4ced-9768-62aedbfe18a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:10.980746 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dadb81-4418-4ced-9768-62aedbfe18a6 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:10.994738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 431/835] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:13:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:11.001522 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dadb81-4418-4ced-9768-62aedbfe18a6 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:11.070170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ed40c-e025-4554-adac-53f467727400 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:11.111375 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694596c-c99d-41a3-be91-16cea10d4114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:11.112431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694596c-c99d-41a3-be91-16cea10d4114 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:11.112788 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694596c-c99d-41a3-be91-16cea10d4114 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:11.113137 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694596c-c99d-41a3-be91-16cea10d4114 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:11.114200 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694596c-c99d-41a3-be91-16cea10d4114 None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=a4a24d5a375945779fa626fae25ac237) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:11.114611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694596c-c99d-41a3-be91-16cea10d4114 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:11.116719 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694596c-c99d-41a3-be91-16cea10d4114 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:11.124771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dadb81-4418-4ced-9768-62aedbfe18a6 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:11.130427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 405/836] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:13:10 2022] GET /identity/v3/role_assignments?user.id=a4a24d5a375945779fa626fae25ac237&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=25516be1485c49538544ea8a596e7d17 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:11.168232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6864f-fb21-4451-9438-9e8e5fe56cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:11.169300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6864f-fb21-4451-9438-9e8e5fe56cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:11.169752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6864f-fb21-4451-9438-9e8e5fe56cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:11.170168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6864f-fb21-4451-9438-9e8e5fe56cd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:11.181884 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba6864f-fb21-4451-9438-9e8e5fe56cd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:11.183854 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba6864f-fb21-4451-9438-9e8e5fe56cd3 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:11.205046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-2ba6864f-fb21-4451-9438-9e8e5fe56cd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:11.205046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:11.205046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:11.205046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:11.205046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:11.205046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:11.205046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:11.205046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:11.205046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:11.205046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:11.205046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:11.205046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:11.205046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:11.205046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:11.205046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:11.206822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:11.206822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:11.206822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:11.206822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:11.206822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:11.206822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:11.206822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:11.206822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:11.206822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:11.206822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:11.206822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:11.206822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:11.206822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:11.206822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:11.206822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:11.206822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:11.206822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:11.207880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:11.207880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:11.207880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:11.207880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:11.214513 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 432/837] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:11.216106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694596c-c99d-41a3-be91-16cea10d4114 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:11.304263 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:11.326849 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694596c-c99d-41a3-be91-16cea10d4114 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:11.364436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c503bbc-7871-4574-a4e6-d2a2d62120f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:11.365146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c503bbc-7871-4574-a4e6-d2a2d62120f4 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:11.365435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c503bbc-7871-4574-a4e6-d2a2d62120f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:11.365710 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c503bbc-7871-4574-a4e6-d2a2d62120f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:11.366462 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c503bbc-7871-4574-a4e6-d2a2d62120f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:11.366790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c503bbc-7871-4574-a4e6-d2a2d62120f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:11.368154 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c503bbc-7871-4574-a4e6-d2a2d62120f4 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:11.380676 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 406/838] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:11.386430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6864f-fb21-4451-9438-9e8e5fe56cd3 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:11.419020 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:11.491286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6864f-fb21-4451-9438-9e8e5fe56cd3 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:11.527325 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831b483-7b4a-4135-a214-0281fde36602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:11.528465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831b483-7b4a-4135-a214-0281fde36602 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:11.528754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831b483-7b4a-4135-a214-0281fde36602 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:11.529029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831b483-7b4a-4135-a214-0281fde36602 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:11.539211 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831b483-7b4a-4135-a214-0281fde36602 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:11.540651 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831b483-7b4a-4135-a214-0281fde36602 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:11.556063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-8831b483-7b4a-4135-a214-0281fde36602 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:13:11.556063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:11.556063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:11.556063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:11.556063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:11.556063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:11.556063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:11.556063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:11.556063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:11.556063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:11.556063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:11.556063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:11.556063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:13:11.556063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:13:11.556063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:13:11.556063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:11.557022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:11.557978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:11.557978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:11.557978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:13:11.557978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:13:11.557978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:13:11.557978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:13:11.557978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:13:11.557978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:13:11.557978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:13:11.557978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:11.561944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 433/839] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Jul 20 15:13:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:11.563327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c503bbc-7871-4574-a4e6-d2a2d62120f4 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:11.671561 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c503bbc-7871-4574-a4e6-d2a2d62120f4 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:11.709435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262dc5b-f3c9-4b61-b82e-7c827160e216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:11.710551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262dc5b-f3c9-4b61-b82e-7c827160e216 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:11.711041 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262dc5b-f3c9-4b61-b82e-7c827160e216 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:11.711577 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262dc5b-f3c9-4b61-b82e-7c827160e216 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:11.712609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262dc5b-f3c9-4b61-b82e-7c827160e216 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:11.713136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262dc5b-f3c9-4b61-b82e-7c827160e216 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:11.714790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262dc5b-f3c9-4b61-b82e-7c827160e216 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:11.717687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:11.734759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 407/840] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Jul 20 15:13:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:11.740348 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831b483-7b4a-4135-a214-0281fde36602 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:11.838870 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:11.849163 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831b483-7b4a-4135-a214-0281fde36602 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:11.884460 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b7a61-786a-4536-9490-52621aa3879f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:11.885175 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b7a61-786a-4536-9490-52621aa3879f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:11.885462 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b7a61-786a-4536-9490-52621aa3879f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:11.885738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b7a61-786a-4536-9490-52621aa3879f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:11.956634 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b7a61-786a-4536-9490-52621aa3879f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:11.958093 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b7a61-786a-4536-9490-52621aa3879f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:12.058801 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:12.062403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 434/841] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:13:11 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:12.071325 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262dc5b-f3c9-4b61-b82e-7c827160e216 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:12.183076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262dc5b-f3c9-4b61-b82e-7c827160e216 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:12.219079 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4061dd2-7ad4-4a19-9d30-664b141b1441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:12.219877 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4061dd2-7ad4-4a19-9d30-664b141b1441 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:12.220160 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4061dd2-7ad4-4a19-9d30-664b141b1441 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:12.220433 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4061dd2-7ad4-4a19-9d30-664b141b1441 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:12.230846 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4061dd2-7ad4-4a19-9d30-664b141b1441 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:12.232290 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4061dd2-7ad4-4a19-9d30-664b141b1441 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:12.250321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-d4061dd2-7ad4-4a19-9d30-664b141b1441 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:12.250321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:12.250321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:12.250321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:12.250321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:12.250321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:12.250321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:12.250321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:12.250321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:12.250321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:12.250321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:12.250321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:12.250321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:12.250321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:12.250321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:12.251269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:12.251269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:12.251269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:12.251269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:12.251269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:12.251269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:12.251269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:12.251269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:12.251269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:12.251269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:12.251269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:12.251269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:12.251269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:12.251269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:12.251269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:12.251269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:12.251269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:12.252459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:12.252459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:12.252459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:12.252459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:12.252459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 408/842] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:12.258002 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b7a61-786a-4536-9490-52621aa3879f None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:12.311682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237beb6-ac01-4eb0-99c9-b37d845fd570 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:12.312035 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237beb6-ac01-4eb0-99c9-b37d845fd570 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:12.312240 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237beb6-ac01-4eb0-99c9-b37d845fd570 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:12.313862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 409/843] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:12.319523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca1cb8-2652-42e5-a89c-63e4198b3106 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:12.319814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca1cb8-2652-42e5-a89c-63e4198b3106 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:12.320092 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca1cb8-2652-42e5-a89c-63e4198b3106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:12.336865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-72ca1cb8-2652-42e5-a89c-63e4198b3106 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:12.338059 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-72ca1cb8-2652-42e5-a89c-63e4198b3106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:12.366365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b7a61-786a-4536-9490-52621aa3879f None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:12.386139 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:12.407981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c90369-c0c9-416e-9c68-8cad9bf21c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:12.408861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c90369-c0c9-416e-9c68-8cad9bf21c56 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:12.409150 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c90369-c0c9-416e-9c68-8cad9bf21c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:12.409426 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c90369-c0c9-416e-9c68-8cad9bf21c56 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:12.410151 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c90369-c0c9-416e-9c68-8cad9bf21c56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:12.410541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c90369-c0c9-416e-9c68-8cad9bf21c56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:12.411977 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c90369-c0c9-416e-9c68-8cad9bf21c56 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:12.418499 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 410/844] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:12.429646 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 435/845] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:12.430992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca1cb8-2652-42e5-a89c-63e4198b3106 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:12.436794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c90369-c0c9-416e-9c68-8cad9bf21c56 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:12.498555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:12.561756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c90369-c0c9-416e-9c68-8cad9bf21c56 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:12.564867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca1cb8-2652-42e5-a89c-63e4198b3106 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:12.604639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd353881-ef4f-4ccf-aa79-8656046eaf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:12.605506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd353881-ef4f-4ccf-aa79-8656046eaf7b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:12.605800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd353881-ef4f-4ccf-aa79-8656046eaf7b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:12.606079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd353881-ef4f-4ccf-aa79-8656046eaf7b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:12.606851 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd353881-ef4f-4ccf-aa79-8656046eaf7b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:12.608506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd353881-ef4f-4ccf-aa79-8656046eaf7b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:12.608961 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65e85b-55a6-4df7-aa0a-83b81408e8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:12.609996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65e85b-55a6-4df7-aa0a-83b81408e8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:12.610562 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65e85b-55a6-4df7-aa0a-83b81408e8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:12.611150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65e85b-55a6-4df7-aa0a-83b81408e8d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:12.619548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65e85b-55a6-4df7-aa0a-83b81408e8d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:12.621365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65e85b-55a6-4df7-aa0a-83b81408e8d5 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:12.621938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-cd353881-ef4f-4ccf-aa79-8656046eaf7b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:13:12.621938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:12.621938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:12.621938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:12.621938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:12.621938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:12.621938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:12.621938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:12.621938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:13:12.621938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:12.621938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:12.621938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:12.621938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:12.621938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:12.621938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:12.621938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:12.621938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:12.623047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:12.623047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:12.623047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:12.623047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:12.623047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:12.623047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:12.623047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:12.623047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:12.623047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:12.623047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:12.623047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:12.623047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:12.623047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:12.623047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:12.623047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:12.623047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:12.623987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:12.623987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:12.623987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:12.623987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:12.623987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:12.623987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:12.623987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:12.623987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:12.623987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:12.623987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:12.623987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:12.623987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:12.623987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:13:12.623987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:12.623987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 436/846] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:13:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:12.624859 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4543f65-3b3a-44bd-a49c-d81451ecc630 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:12.624859 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4543f65-3b3a-44bd-a49c-d81451ecc630 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:12.624983 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4543f65-3b3a-44bd-a49c-d81451ecc630 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:12.631045 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 437/847] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:12.631045 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4543f65-3b3a-44bd-a49c-d81451ecc630 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:12.642097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-4d65e85b-55a6-4df7-aa0a-83b81408e8d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:12.642097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:12.642097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:12.642097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:12.642097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:12.642097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:12.642097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:12.642097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:12.642097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:12.642097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:12.642097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:12.642097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:12.642097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:13:12.642097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:13:12.642097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:13:12.643347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:13:12.643347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:12.643347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:12.643347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:12.643347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:12.643347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:12.643347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:12.643347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:12.643347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:12.643347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:12.643347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:12.643347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:12.643347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:12.643347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:12.643347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:12.643347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:12.643347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:12.644908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:13:12.644908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:13:12.644908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:13:12.644908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:13:12.644908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:13:12.644908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:13:12.644908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:12.644908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:12.644908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 411/848] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:12.644908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ecb0d-8d9f-4a7f-962c-3c1aa8abb464 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:12.644908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ecb0d-8d9f-4a7f-962c-3c1aa8abb464 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:12.645552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ecb0d-8d9f-4a7f-962c-3c1aa8abb464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:12.665712 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-883ecb0d-8d9f-4a7f-962c-3c1aa8abb464 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:12.666937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-883ecb0d-8d9f-4a7f-962c-3c1aa8abb464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:12.696278 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:12.734596 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 412/849] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:12.739397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ecb0d-8d9f-4a7f-962c-3c1aa8abb464 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:12.744625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4543f65-3b3a-44bd-a49c-d81451ecc630 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:12.765968 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:12.786104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c54d32-bccc-4494-a0e6-3cc2008798d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:12.786878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c54d32-bccc-4494-a0e6-3cc2008798d2 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:12.787159 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c54d32-bccc-4494-a0e6-3cc2008798d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:12.787433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c54d32-bccc-4494-a0e6-3cc2008798d2 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:12.788135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c54d32-bccc-4494-a0e6-3cc2008798d2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:12.788429 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c54d32-bccc-4494-a0e6-3cc2008798d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:12.789795 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c54d32-bccc-4494-a0e6-3cc2008798d2 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:12.798815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 438/850] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:13:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:12.798815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c54d32-bccc-4494-a0e6-3cc2008798d2 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:12.854044 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ecb0d-8d9f-4a7f-962c-3c1aa8abb464 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:12.889391 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da606436-2ccb-4845-8f4a-0a322722117c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:12.889979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da606436-2ccb-4845-8f4a-0a322722117c None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:12.890289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da606436-2ccb-4845-8f4a-0a322722117c None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:12.890568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da606436-2ccb-4845-8f4a-0a322722117c None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:12.891354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da606436-2ccb-4845-8f4a-0a322722117c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:12.891669 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da606436-2ccb-4845-8f4a-0a322722117c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:12.893086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da606436-2ccb-4845-8f4a-0a322722117c None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:12.899461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c54d32-bccc-4494-a0e6-3cc2008798d2 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:12.909990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 413/851] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:13:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:12.909990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da606436-2ccb-4845-8f4a-0a322722117c None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:12.938306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:12.938710 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4dd7a2-7f5e-468f-be36-5d779eea8c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:12.940006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dd7a2-7f5e-468f-be36-5d779eea8c77 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:12.940299 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dd7a2-7f5e-468f-be36-5d779eea8c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:12.940584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dd7a2-7f5e-468f-be36-5d779eea8c77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:12.951165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4dd7a2-7f5e-468f-be36-5d779eea8c77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:13:12.958878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-9a4dd7a2-7f5e-468f-be36-5d779eea8c77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:13:12.958878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:12.958878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:12.958878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:12.958878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:12.958878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:12.958878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:12.958878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:12.958878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:12.958878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:12.958878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:12.958878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:12.958878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:13:12.958878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:13:12.958878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:13:12.958878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:13:12.960035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:13:12.960035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:13:12.960035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:13:12.960035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:13:12.960035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:12.960035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:12.960035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:12.960035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:12.960035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:12.960035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:12.960035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:12.960035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:12.960035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:12.960035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:12.960035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:12.960035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:12.960908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:12.960908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:12.960908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:12.960908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:12.960908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:13:12.960908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:13:12.960908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:13:12.960908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:13:12.960908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:13:12.960908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:13:12.960908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:13:12.960908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:12.960908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 439/852] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:13:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:12.960908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4dd7a2-7f5e-468f-be36-5d779eea8c77 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:13.009916 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:13.021619 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da606436-2ccb-4845-8f4a-0a322722117c None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:13.055980 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9c1da-6823-43c8-aaf0-59b01e1e015b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:13.056635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9c1da-6823-43c8-aaf0-59b01e1e015b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:13.056913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9c1da-6823-43c8-aaf0-59b01e1e015b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:13.057185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9c1da-6823-43c8-aaf0-59b01e1e015b None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:13.057892 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9c1da-6823-43c8-aaf0-59b01e1e015b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:13.058199 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9c1da-6823-43c8-aaf0-59b01e1e015b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:13.059567 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9c1da-6823-43c8-aaf0-59b01e1e015b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:13.074303 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 414/853] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:13:12 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:13.074303 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9c1da-6823-43c8-aaf0-59b01e1e015b None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:13.086438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4dd7a2-7f5e-468f-be36-5d779eea8c77 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:13.131738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85959ca1-7c27-4c0b-93fd-5b978003da11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:13.132434 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85959ca1-7c27-4c0b-93fd-5b978003da11 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:13.132719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85959ca1-7c27-4c0b-93fd-5b978003da11 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:13.132997 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85959ca1-7c27-4c0b-93fd-5b978003da11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:13.133875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85959ca1-7c27-4c0b-93fd-5b978003da11 None admin] RBAC: Adding query filter params (role.id=d820b1d3fa4144aca575c04645c099bc, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=4e75a82cc7e94702ba0080fc65d6eef3) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:13.134190 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85959ca1-7c27-4c0b-93fd-5b978003da11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:13.135733 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85959ca1-7c27-4c0b-93fd-5b978003da11 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:13.143287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 440/854] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:13:12 2022] GET /identity/v3/role_assignments?user.id=4e75a82cc7e94702ba0080fc65d6eef3&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=d820b1d3fa4144aca575c04645c099bc => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:13.209736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:13.221837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9c1da-6823-43c8-aaf0-59b01e1e015b None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:13.264049 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278dd88-24ca-413b-9072-317c55d58770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:13.265133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278dd88-24ca-413b-9072-317c55d58770 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:13.265615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278dd88-24ca-413b-9072-317c55d58770 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:13.266109 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278dd88-24ca-413b-9072-317c55d58770 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:13.267098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278dd88-24ca-413b-9072-317c55d58770 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:13.267605 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278dd88-24ca-413b-9072-317c55d58770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:13.269283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278dd88-24ca-413b-9072-317c55d58770 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:13.279982 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 415/855] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:13:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:13.285579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85959ca1-7c27-4c0b-93fd-5b978003da11 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:13.329001 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701fb5d-8811-4a09-93ad-a41ec103bcad None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:13.329654 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701fb5d-8811-4a09-93ad-a41ec103bcad None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:13.330150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701fb5d-8811-4a09-93ad-a41ec103bcad None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:13.332719 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 416/856] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:13:13.339117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103bd08-2ed7-4139-a0fa-25448adf47d7 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:13.339413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103bd08-2ed7-4139-a0fa-25448adf47d7 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:13.339694 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103bd08-2ed7-4139-a0fa-25448adf47d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:13.362749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-2103bd08-2ed7-4139-a0fa-25448adf47d7 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:13.363995 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-2103bd08-2ed7-4139-a0fa-25448adf47d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:13.415394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85959ca1-7c27-4c0b-93fd-5b978003da11 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:13.417194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:13.449740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 417/857] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:13:13.460240 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376948c-97bc-45f6-bcf6-09e08a9c7dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:13.461040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376948c-97bc-45f6-bcf6-09e08a9c7dbc None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:13.461322 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376948c-97bc-45f6-bcf6-09e08a9c7dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:13.461590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376948c-97bc-45f6-bcf6-09e08a9c7dbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:13.472941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376948c-97bc-45f6-bcf6-09e08a9c7dbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:13.474450 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376948c-97bc-45f6-bcf6-09e08a9c7dbc None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:13.475441 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103bd08-2ed7-4139-a0fa-25448adf47d7 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:13.494060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-3376948c-97bc-45f6-bcf6-09e08a9c7dbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:13.494060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:13.494060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:13.494060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:13.494060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:13.494060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:13.494060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:13.494060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:13.494060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:13.494060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:13.494060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:13.494060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:13.494060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:13.494060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:13.494060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:13.495113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:13.495113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:13.495113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:13.495113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:13.495113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:13.495113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:13.495113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:13.495113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:13.495113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:13.495113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:13.495113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:13.495113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:13.495113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:13.495113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:13.495113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:13.495113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:13.495113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:13.496057 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:13.496057 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:13.496057 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:13.496057 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:13.496057 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 441/858] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:13.500876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376948c-97bc-45f6-bcf6-09e08a9c7dbc None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:13.572074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:13.592050 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103bd08-2ed7-4139-a0fa-25448adf47d7 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:13.618405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376948c-97bc-45f6-bcf6-09e08a9c7dbc None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:13.632505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c229c4-190f-4556-b75f-2f3642fbddd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:13.633474 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c229c4-190f-4556-b75f-2f3642fbddd2 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:13.633858 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c229c4-190f-4556-b75f-2f3642fbddd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:13.634135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c229c4-190f-4556-b75f-2f3642fbddd2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:13.634876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c229c4-190f-4556-b75f-2f3642fbddd2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:13.636549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c229c4-190f-4556-b75f-2f3642fbddd2 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:13.649932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-f7c229c4-190f-4556-b75f-2f3642fbddd2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:13:13.649932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:13.649932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:13.649932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:13.649932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:13.649932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:13.649932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:13.649932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:13.649932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:13:13.649932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:13.649932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:13.649932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:13.649932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:13.649932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:13.649932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:13.649932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:13.650976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:13.650976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:13.650976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:13.650976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:13.650976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:13.650976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:13.650976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:13.650976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:13.650976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:13.650976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:13.650976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:13.650976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:13.650976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:13.650976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:13.650976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:13.650976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:13.652523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:13.652523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:13.652523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:13.652523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:13.652523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:13.652523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:13.652523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:13.652523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:13.652523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:13.652523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:13.652523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:13.652523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:13.652523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:13.652523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:13:13.652523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:13.652523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 418/859] 10.222.0.78 () {56 vars in 1132 bytes} [Wed Jul 20 15:13:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:13.659932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c229c4-190f-4556-b75f-2f3642fbddd2 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:13.660648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40edb76-7063-4996-b560-2ee1b96f55ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:13.661419 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40edb76-7063-4996-b560-2ee1b96f55ae None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:13.661802 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40edb76-7063-4996-b560-2ee1b96f55ae None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:13.662168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40edb76-7063-4996-b560-2ee1b96f55ae None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:13.663019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40edb76-7063-4996-b560-2ee1b96f55ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:13.663418 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40edb76-7063-4996-b560-2ee1b96f55ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:13.666295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40edb76-7063-4996-b560-2ee1b96f55ae None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:13.681969 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 442/860] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:13.687798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40edb76-7063-4996-b560-2ee1b96f55ae None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:13.704049 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:13.773553 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c229c4-190f-4556-b75f-2f3642fbddd2 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:13.791372 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40edb76-7063-4996-b560-2ee1b96f55ae None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:13.811326 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c548a-2854-4e21-90c4-1d10cbe8b378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:13.812213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c548a-2854-4e21-90c4-1d10cbe8b378 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:13.812641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c548a-2854-4e21-90c4-1d10cbe8b378 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:13.813060 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c548a-2854-4e21-90c4-1d10cbe8b378 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:13.813909 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c548a-2854-4e21-90c4-1d10cbe8b378 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:13.814428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c548a-2854-4e21-90c4-1d10cbe8b378 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:13.816477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c548a-2854-4e21-90c4-1d10cbe8b378 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:13.821436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 419/861] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Jul 20 15:13:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:13.826629 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c548a-2854-4e21-90c4-1d10cbe8b378 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:13.841082 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:13.842176 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b2602-63a4-42e5-a66c-7d070c4d622c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:13.843098 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b2602-63a4-42e5-a66c-7d070c4d622c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:13.843455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b2602-63a4-42e5-a66c-7d070c4d622c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:13.843806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b2602-63a4-42e5-a66c-7d070c4d622c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:13.853538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b2602-63a4-42e5-a66c-7d070c4d622c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:13.856363 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b2602-63a4-42e5-a66c-7d070c4d622c None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:13.875437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-353b2602-63a4-42e5-a66c-7d070c4d622c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:13:13.875437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:13.875437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:13.875437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:13.875437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:13.875437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:13.875437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:13.875437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:13.875437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:13.875437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:13.875437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:13.875437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:13.875437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:13:13.875437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:13:13.875437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:13:13.875437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:13.876629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:13.878748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:13.878748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:13.878748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:13:13.878748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:13:13.878748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:13:13.878748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:13:13.878748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:13:13.878748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:13:13.878748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 15:13:13.878748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:13.879486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 443/862] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:13:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:13.887355 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b2602-63a4-42e5-a66c-7d070c4d622c None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:13.939695 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c548a-2854-4e21-90c4-1d10cbe8b378 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:13.975594 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c04c4e-6a68-426e-89dc-d1f6475684df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:13.976945 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c04c4e-6a68-426e-89dc-d1f6475684df None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:13.977581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c04c4e-6a68-426e-89dc-d1f6475684df None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:13.978189 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c04c4e-6a68-426e-89dc-d1f6475684df None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:13.979701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c04c4e-6a68-426e-89dc-d1f6475684df None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:13.980346 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c04c4e-6a68-426e-89dc-d1f6475684df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:13.982647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c04c4e-6a68-426e-89dc-d1f6475684df None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:13.984739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:13.989194 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 420/863] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Jul 20 15:13:13 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:14.020628 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b2602-63a4-42e5-a66c-7d070c4d622c None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:14.062574 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094d1ba-c699-4a8c-9526-8267b4618092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:14.063260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094d1ba-c699-4a8c-9526-8267b4618092 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:14.063541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094d1ba-c699-4a8c-9526-8267b4618092 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:14.063816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094d1ba-c699-4a8c-9526-8267b4618092 None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:14.064672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094d1ba-c699-4a8c-9526-8267b4618092 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:14.065018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094d1ba-c699-4a8c-9526-8267b4618092 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:14.067053 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094d1ba-c699-4a8c-9526-8267b4618092 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:14.087865 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 444/864] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Jul 20 15:13:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:14.094292 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c04c4e-6a68-426e-89dc-d1f6475684df None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:14.181566 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:14.206167 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c04c4e-6a68-426e-89dc-d1f6475684df None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:14.241432 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76769716-e004-4d5a-ab7a-8dbf2199f71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:14.242519 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76769716-e004-4d5a-ab7a-8dbf2199f71a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:14.242803 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76769716-e004-4d5a-ab7a-8dbf2199f71a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:14.243081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76769716-e004-4d5a-ab7a-8dbf2199f71a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:14.315327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76769716-e004-4d5a-ab7a-8dbf2199f71a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:14.317228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76769716-e004-4d5a-ab7a-8dbf2199f71a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:14.402807 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:14.427733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 421/865] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:13:14 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:14.433337 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094d1ba-c699-4a8c-9526-8267b4618092 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:14.542613 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094d1ba-c699-4a8c-9526-8267b4618092 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:14.572494 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:14.577543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9bacf-2da5-4843-a73e-0139ab85af44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:14.578375 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9bacf-2da5-4843-a73e-0139ab85af44 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:14.578668 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9bacf-2da5-4843-a73e-0139ab85af44 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:14.578954 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9bacf-2da5-4843-a73e-0139ab85af44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:14.589672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9bacf-2da5-4843-a73e-0139ab85af44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:14.591145 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9bacf-2da5-4843-a73e-0139ab85af44 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:14.609777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-3cb9bacf-2da5-4843-a73e-0139ab85af44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:14.609777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:14.609777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:14.609777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:14.609777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:14.609777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:14.609777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:14.609777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:14.609777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:14.609777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:14.609777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:14.609777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:14.609777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:14.609777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:14.609777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:14.610722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:14.610722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:14.610722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:14.610722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:14.610722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:14.610722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:14.610722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:14.610722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:14.610722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:14.610722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:14.610722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:14.610722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:14.610722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:14.610722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:14.610722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:14.610722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:14.610722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:14.611622 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:14.611622 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:14.611622 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:14.611622 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:14.611622 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 445/866] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:14.616941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76769716-e004-4d5a-ab7a-8dbf2199f71a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:14.718878 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76769716-e004-4d5a-ab7a-8dbf2199f71a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:14.759008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbea1b-2df9-4df0-8890-bb970d588112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:14.759676 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbea1b-2df9-4df0-8890-bb970d588112 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:14.759958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbea1b-2df9-4df0-8890-bb970d588112 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:14.760233 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbea1b-2df9-4df0-8890-bb970d588112 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:14.760961 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadbea1b-2df9-4df0-8890-bb970d588112 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:14.761270 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadbea1b-2df9-4df0-8890-bb970d588112 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:14.762661 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadbea1b-2df9-4df0-8890-bb970d588112 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:14.775189 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 422/867] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:14.780632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9bacf-2da5-4843-a73e-0139ab85af44 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:14.883134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9bacf-2da5-4843-a73e-0139ab85af44 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:14.919307 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651d718-76bd-4cfa-899f-f243f220301f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:14.920124 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651d718-76bd-4cfa-899f-f243f220301f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:14.920411 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651d718-76bd-4cfa-899f-f243f220301f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:14.920690 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651d718-76bd-4cfa-899f-f243f220301f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:14.928470 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651d718-76bd-4cfa-899f-f243f220301f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:14.929924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651d718-76bd-4cfa-899f-f243f220301f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:14.943910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-a651d718-76bd-4cfa-899f-f243f220301f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:14.943910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:14.943910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:14.943910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:14.943910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:14.943910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:14.943910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:14.943910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:14.943910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:14.943910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:14.943910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:14.943910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:14.943910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:13:14.943910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:13:14.943910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:13:14.944824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:13:14.944824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:14.944824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:14.944824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:14.944824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:14.944824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:14.944824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:14.944824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:14.944824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:14.944824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:14.944824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:14.944824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:14.944824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:14.944824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:14.944824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:14.944824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:14.944824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:14.945739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:13:14.945739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:13:14.945739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:13:14.945739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:13:14.945739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:13:14.945739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:13:14.945739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:14.945739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:14.945739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 446/868] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:14.951787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbea1b-2df9-4df0-8890-bb970d588112 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:14.990086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:15.059330 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbea1b-2df9-4df0-8890-bb970d588112 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:15.101667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3074693-4af8-47b2-933a-3bb4ab90f63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:15.102637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3074693-4af8-47b2-933a-3bb4ab90f63e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:15.103134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3074693-4af8-47b2-933a-3bb4ab90f63e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:15.103605 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3074693-4af8-47b2-933a-3bb4ab90f63e None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:15.104594 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3074693-4af8-47b2-933a-3bb4ab90f63e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:15.105100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3074693-4af8-47b2-933a-3bb4ab90f63e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:15.106768 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3074693-4af8-47b2-933a-3bb4ab90f63e None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:15.119704 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 423/869] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:13:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:15.126305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651d718-76bd-4cfa-899f-f243f220301f None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:15.177578 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:15.235528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651d718-76bd-4cfa-899f-f243f220301f None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:15.277376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652034f-04fc-43da-b81a-6e41c878b013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:15.278053 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652034f-04fc-43da-b81a-6e41c878b013 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:15.278374 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652034f-04fc-43da-b81a-6e41c878b013 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:15.278657 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652034f-04fc-43da-b81a-6e41c878b013 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:15.279500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652034f-04fc-43da-b81a-6e41c878b013 None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=ed0d719c0eb04a3b8a228c6ce70fdf98) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:15.279817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652034f-04fc-43da-b81a-6e41c878b013 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:15.281268 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652034f-04fc-43da-b81a-6e41c878b013 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:15.285272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3ca47-5d8b-4156-9c86-58792b63bbe0 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:15.285600 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3ca47-5d8b-4156-9c86-58792b63bbe0 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:15.285947 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3ca47-5d8b-4156-9c86-58792b63bbe0 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:15.287431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 424/870] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:13:15.289879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 447/871] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:13:15 2022] GET /identity/v3/role_assignments?user.id=ed0d719c0eb04a3b8a228c6ce70fdf98&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=25516be1485c49538544ea8a596e7d17 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:15.294351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d10652-320b-4532-88ea-cf957d672a40 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:15.294351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d10652-320b-4532-88ea-cf957d672a40 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:15.294351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d10652-320b-4532-88ea-cf957d672a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:15.311848 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-34d10652-320b-4532-88ea-cf957d672a40 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:15.313547 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-34d10652-320b-4532-88ea-cf957d672a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:15.391978 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 425/872] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:13:15.415786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652034f-04fc-43da-b81a-6e41c878b013 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:15.519763 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cbf47-25eb-435c-b812-0ec4f08a4af1 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:15.520042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cbf47-25eb-435c-b812-0ec4f08a4af1 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:15.520444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cbf47-25eb-435c-b812-0ec4f08a4af1 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:15.521999 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 426/873] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:13:15.526976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de83487-ebf8-4b4f-a70f-02b33a3ccca7 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:15.527204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de83487-ebf8-4b4f-a70f-02b33a3ccca7 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:15.528044 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de83487-ebf8-4b4f-a70f-02b33a3ccca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:15.531456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652034f-04fc-43da-b81a-6e41c878b013 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:15.543342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:15.545100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-2de83487-ebf8-4b4f-a70f-02b33a3ccca7 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:15.546952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-2de83487-ebf8-4b4f-a70f-02b33a3ccca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:15.567226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281e591-cc77-4576-ba43-ca3865e9310f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:15.567907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281e591-cc77-4576-ba43-ca3865e9310f None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:15.568193 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281e591-cc77-4576-ba43-ca3865e9310f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:15.568473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281e591-cc77-4576-ba43-ca3865e9310f None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:15.569202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281e591-cc77-4576-ba43-ca3865e9310f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:15.569734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:15.570606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281e591-cc77-4576-ba43-ca3865e9310f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:15.623053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 427/874] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:13:15.648386 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de83487-ebf8-4b4f-a70f-02b33a3ccca7 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:15.687702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 448/875] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Jul 20 15:13:15 2022] POST /identity/v3/services => generated 251 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:13:15.773784 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de83487-ebf8-4b4f-a70f-02b33a3ccca7 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:15.828854 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b052395-4f94-4b9b-b743-9602e7beb591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:15.830104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b052395-4f94-4b9b-b743-9602e7beb591 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:15.830693 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b052395-4f94-4b9b-b743-9602e7beb591 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:15.831179 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b052395-4f94-4b9b-b743-9602e7beb591 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:15.844509 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b052395-4f94-4b9b-b743-9602e7beb591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:13:15.854008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-5b052395-4f94-4b9b-b743-9602e7beb591 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:13:15.854008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:15.854008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:15.854008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:15.854008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:15.854008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:15.854008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:15.854008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:15.854008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:15.854008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:15.854008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:15.854008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:15.854008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:13:15.854008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:13:15.854008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:13:15.854008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:13:15.855026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:13:15.855026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:13:15.855026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:13:15.855026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:13:15.855026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:15.855026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:15.855026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:15.855026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:15.855026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:15.855026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:15.855026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:15.855026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:15.855026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:15.855026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:15.855026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:15.855026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:15.855913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:15.855913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:15.855913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:15.855913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:15.855913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:13:15.855913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:13:15.855913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:13:15.855913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:13:15.855913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:13:15.855913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:13:15.855913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:13:15.855913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:15.857818 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 428/876] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Jul 20 15:13:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:15.863055 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281e591-cc77-4576-ba43-ca3865e9310f None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:15.969041 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281e591-cc77-4576-ba43-ca3865e9310f None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:16.004472 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398cde4-3099-4881-934e-193c44ceb611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:16.005149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398cde4-3099-4881-934e-193c44ceb611 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:16.005434 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398cde4-3099-4881-934e-193c44ceb611 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:16.005711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398cde4-3099-4881-934e-193c44ceb611 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:16.006490 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398cde4-3099-4881-934e-193c44ceb611 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:16.006803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398cde4-3099-4881-934e-193c44ceb611 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:16.008184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398cde4-3099-4881-934e-193c44ceb611 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:16.016878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 449/877] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Jul 20 15:13:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:16.022824 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b052395-4f94-4b9b-b743-9602e7beb591 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:16.036163 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:16.137009 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b052395-4f94-4b9b-b743-9602e7beb591 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:16.164164 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845b7ba-e0fc-4123-ab74-8e0c0686d9e0 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:16.164528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845b7ba-e0fc-4123-ab74-8e0c0686d9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:16.164847 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845b7ba-e0fc-4123-ab74-8e0c0686d9e0 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:16.166697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 450/878] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:16.171709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a16ac3-f8ca-487a-8929-0bc02ab70c0d None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:16.172024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a16ac3-f8ca-487a-8929-0bc02ab70c0d None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:16.172302 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a16ac3-f8ca-487a-8929-0bc02ab70c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:16.173180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0331680-86b8-4b9f-8894-b344ce9ad486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:16.174053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0331680-86b8-4b9f-8894-b344ce9ad486 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:16.174798 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0331680-86b8-4b9f-8894-b344ce9ad486 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:16.175081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0331680-86b8-4b9f-8894-b344ce9ad486 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:16.187778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0331680-86b8-4b9f-8894-b344ce9ad486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:16.189235 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0331680-86b8-4b9f-8894-b344ce9ad486 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:16.189750 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-e9a16ac3-f8ca-487a-8929-0bc02ab70c0d None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:16.191155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e9a16ac3-f8ca-487a-8929-0bc02ab70c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:16.211941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-d0331680-86b8-4b9f-8894-b344ce9ad486 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:16.211941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:16.211941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:16.211941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:16.211941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:16.211941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:16.211941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:16.211941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:16.211941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:16.211941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:16.211941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:16.211941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:16.211941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:16.211941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:16.211941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:16.213015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:16.213015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:16.213015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:16.213015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:16.213015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:16.213015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:16.213015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:16.213015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:16.213015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:16.213015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:16.213015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:16.213015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:16.213015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:16.213015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:16.213015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:16.213015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:16.213015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:16.214565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:16.214565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:16.214565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:16.214565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:16.214565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 429/879] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:16.220728 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:16.221631 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0331680-86b8-4b9f-8894-b344ce9ad486 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:16.275180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 451/880] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:16 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:13:16.314424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a16ac3-f8ca-487a-8929-0bc02ab70c0d None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:16.314620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:16.348281 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0331680-86b8-4b9f-8894-b344ce9ad486 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:16.391996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f90fe-fc1f-4535-b7c5-5a9c2f236b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:16.393157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f90fe-fc1f-4535-b7c5-5a9c2f236b44 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:16.393451 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f90fe-fc1f-4535-b7c5-5a9c2f236b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:16.393732 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f90fe-fc1f-4535-b7c5-5a9c2f236b44 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:16.394765 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f90fe-fc1f-4535-b7c5-5a9c2f236b44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:16.395082 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f90fe-fc1f-4535-b7c5-5a9c2f236b44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:16.397156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f90fe-fc1f-4535-b7c5-5a9c2f236b44 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:16.403280 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a16ac3-f8ca-487a-8929-0bc02ab70c0d None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:16.414219 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 430/881] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:16.419571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f90fe-fc1f-4535-b7c5-5a9c2f236b44 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:16.446094 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abc7d8-a251-44b3-bd67-9c4f2cef84df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:16.447208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abc7d8-a251-44b3-bd67-9c4f2cef84df None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:16.447730 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abc7d8-a251-44b3-bd67-9c4f2cef84df None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:16.448205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abc7d8-a251-44b3-bd67-9c4f2cef84df None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:16.449268 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abc7d8-a251-44b3-bd67-9c4f2cef84df None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:16.450871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abc7d8-a251-44b3-bd67-9c4f2cef84df None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:16.524879 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f90fe-fc1f-4535-b7c5-5a9c2f236b44 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:16.548048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:16.566436 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 452/882] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Jul 20 15:13:16 2022] POST /identity/v3/services => generated 269 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:13:16.577047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af8da9-8e21-4ec9-9b5a-ae83ee1b4f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:16.582188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af8da9-8e21-4ec9-9b5a-ae83ee1b4f08 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:16.582188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af8da9-8e21-4ec9-9b5a-ae83ee1b4f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:16.582188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af8da9-8e21-4ec9-9b5a-ae83ee1b4f08 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:16.588714 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af8da9-8e21-4ec9-9b5a-ae83ee1b4f08 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:16.590796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af8da9-8e21-4ec9-9b5a-ae83ee1b4f08 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:16.608419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-61af8da9-8e21-4ec9-9b5a-ae83ee1b4f08 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:13:16.608419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:16.608419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:16.608419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:16.608419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:16.608419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:16.608419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:16.608419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:16.608419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:16.608419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:16.608419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:16.608419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:16.608419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:13:16.608419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:13:16.608419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:13:16.608419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:16.609490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:16.610580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:16.610580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:16.610580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:13:16.610580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:13:16.610580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:13:16.610580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:13:16.610580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:13:16.610580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:13:16.610580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:13:16.610580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:16.618683 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 431/883] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Jul 20 15:13:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:16.620738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abc7d8-a251-44b3-bd67-9c4f2cef84df None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:16.648083 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:16.736260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abc7d8-a251-44b3-bd67-9c4f2cef84df None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:16.773071 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351437d2-0bb0-49fc-9b33-007ca131453d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:16.774127 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351437d2-0bb0-49fc-9b33-007ca131453d None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:16.774653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351437d2-0bb0-49fc-9b33-007ca131453d None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:16.775214 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351437d2-0bb0-49fc-9b33-007ca131453d None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:16.776425 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351437d2-0bb0-49fc-9b33-007ca131453d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:16.776963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351437d2-0bb0-49fc-9b33-007ca131453d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:16.778684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351437d2-0bb0-49fc-9b33-007ca131453d None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:16.802053 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 453/884] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Jul 20 15:13:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:16.803708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af8da9-8e21-4ec9-9b5a-ae83ee1b4f08 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:16.912601 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af8da9-8e21-4ec9-9b5a-ae83ee1b4f08 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:16.949963 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddaa44-d755-473a-b53d-6680a6d814c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:16.950653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddaa44-d755-473a-b53d-6680a6d814c4 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:16.950939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddaa44-d755-473a-b53d-6680a6d814c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:16.951215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddaa44-d755-473a-b53d-6680a6d814c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:17.000223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:17.023964 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddaa44-d755-473a-b53d-6680a6d814c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:17.025495 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddaa44-d755-473a-b53d-6680a6d814c4 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:17.132908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 432/885] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:13:16 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:17.138467 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351437d2-0bb0-49fc-9b33-007ca131453d None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:17.222621 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:17.244348 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351437d2-0bb0-49fc-9b33-007ca131453d None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:17.280157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913278d6-e30b-41bd-8f0a-bbd98725e9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:17.280971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913278d6-e30b-41bd-8f0a-bbd98725e9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:17.281257 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913278d6-e30b-41bd-8f0a-bbd98725e9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:17.281544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913278d6-e30b-41bd-8f0a-bbd98725e9b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:17.292693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913278d6-e30b-41bd-8f0a-bbd98725e9b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:17.294180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913278d6-e30b-41bd-8f0a-bbd98725e9b3 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:17.312748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-913278d6-e30b-41bd-8f0a-bbd98725e9b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:17.312748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:17.312748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:17.312748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:17.312748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:17.312748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:17.312748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:17.312748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:17.312748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:17.312748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:17.312748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:17.312748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:17.312748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:17.312748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:17.312748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:17.313695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:17.313695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:17.313695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:17.313695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:17.313695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:17.313695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:17.313695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:17.313695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:17.313695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:17.313695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:17.313695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:17.313695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:17.313695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:17.313695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:17.313695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:17.313695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:17.313695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:17.314677 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:17.314677 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:17.314677 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:17.314677 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:17.318436 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 454/886] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:17.320010 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddaa44-d755-473a-b53d-6680a6d814c4 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:17.401465 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:17.423971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddaa44-d755-473a-b53d-6680a6d814c4 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:17.463381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5c067-2232-4e7f-887f-cbed7f59f7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:17.464046 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5c067-2232-4e7f-887f-cbed7f59f7bd None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:17.464327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5c067-2232-4e7f-887f-cbed7f59f7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:17.464601 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5c067-2232-4e7f-887f-cbed7f59f7bd None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:17.465322 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5c067-2232-4e7f-887f-cbed7f59f7bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:17.465627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5c067-2232-4e7f-887f-cbed7f59f7bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:17.467018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5c067-2232-4e7f-887f-cbed7f59f7bd None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:17.479635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 433/887] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:17.484941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913278d6-e30b-41bd-8f0a-bbd98725e9b3 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:17.508458 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9a1cc-e28c-49c2-8889-c69ed69c516e None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:17.508784 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9a1cc-e28c-49c2-8889-c69ed69c516e None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:17.509014 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9a1cc-e28c-49c2-8889-c69ed69c516e None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:17.510649 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 434/888] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:13:17.515493 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d8454-d7ca-4a82-81da-2b0a732d17de None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:17.515779 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d8454-d7ca-4a82-81da-2b0a732d17de None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:17.516060 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d8454-d7ca-4a82-81da-2b0a732d17de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:17.533379 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-5f5d8454-d7ca-4a82-81da-2b0a732d17de None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:17.535204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5f5d8454-d7ca-4a82-81da-2b0a732d17de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:17.610018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913278d6-e30b-41bd-8f0a-bbd98725e9b3 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:17.610662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 435/889] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:13:17.638164 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d8454-d7ca-4a82-81da-2b0a732d17de None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:17.647072 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:17.654883 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cb819-9e46-4879-87b6-5c189fa48ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:17.655965 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cb819-9e46-4879-87b6-5c189fa48ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:17.656250 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cb819-9e46-4879-87b6-5c189fa48ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:17.656528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cb819-9e46-4879-87b6-5c189fa48ab4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:17.665768 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3cb819-9e46-4879-87b6-5c189fa48ab4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:17.667629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3cb819-9e46-4879-87b6-5c189fa48ab4 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:17.687520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-fa3cb819-9e46-4879-87b6-5c189fa48ab4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:17.687520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:17.687520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:17.687520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:17.687520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:17.687520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:17.687520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:17.687520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:17.687520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:17.687520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:17.687520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:17.687520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:17.687520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:13:17.687520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:13:17.687520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:13:17.688790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:13:17.688790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:17.688790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:17.688790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:17.688790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:17.688790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:17.688790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:17.688790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:17.688790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:17.688790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:17.688790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:17.688790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:17.688790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:17.688790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:17.688790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:17.688790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:17.688790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:17.690648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:13:17.690648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:13:17.690648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:13:17.690648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:13:17.690648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:13:17.690648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:13:17.690648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:17.690648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:17.690648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 455/890] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:17.695306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cb819-9e46-4879-87b6-5c189fa48ab4 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:17.720377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:17.763480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d8454-d7ca-4a82-81da-2b0a732d17de None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:17.801602 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4473d8-77f3-4b0f-9d4c-e3ac5ce57fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:17.802749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4473d8-77f3-4b0f-9d4c-e3ac5ce57fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:17.803198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4473d8-77f3-4b0f-9d4c-e3ac5ce57fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:17.803622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4473d8-77f3-4b0f-9d4c-e3ac5ce57fe6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:17.804473 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4473d8-77f3-4b0f-9d4c-e3ac5ce57fe6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:17.806480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4473d8-77f3-4b0f-9d4c-e3ac5ce57fe6 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:17.806591 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cb819-9e46-4879-87b6-5c189fa48ab4 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:17.819142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-3f4473d8-77f3-4b0f-9d4c-e3ac5ce57fe6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:13:17.819142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:17.819142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:17.819142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:17.819142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:17.819142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:17.819142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:17.819142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:17.819142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:13:17.819142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:17.819142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:17.819142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:17.819142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:17.819142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:17.819142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:17.819142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:17.819142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:17.820255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:17.820255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:17.820255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:17.820255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:17.820255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:17.820255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:17.820255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:17.820255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:17.820255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:17.820255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:17.820255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:17.820255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:17.820255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:17.820255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:17.820255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:17.820255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:17.821173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:17.821173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:17.821173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:17.821173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:17.821173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:17.821173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:17.821173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:17.821173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:17.821173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:17.821173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:17.821173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:17.821173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:17.821173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:13:17.821173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:17.828750 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 436/891] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Jul 20 15:13:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:17.829831 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4473d8-77f3-4b0f-9d4c-e3ac5ce57fe6 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:17.842041 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d997f4f-4ba3-449c-b97b-95a8f082f1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:17.842841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d997f4f-4ba3-449c-b97b-95a8f082f1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:17.843129 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d997f4f-4ba3-449c-b97b-95a8f082f1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:17.843408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d997f4f-4ba3-449c-b97b-95a8f082f1f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:17.844392 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d997f4f-4ba3-449c-b97b-95a8f082f1f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:17.844510 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d997f4f-4ba3-449c-b97b-95a8f082f1f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:17.845940 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d997f4f-4ba3-449c-b97b-95a8f082f1f3 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:17.859020 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 456/892] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:13:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:17.864766 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d997f4f-4ba3-449c-b97b-95a8f082f1f3 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:17.919648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:17.952379 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4473d8-77f3-4b0f-9d4c-e3ac5ce57fe6 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:17.972593 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d997f4f-4ba3-449c-b97b-95a8f082f1f3 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:17.982838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:17.987981 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528cc08-9c15-4c27-b2ba-84a1484e9851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:17.988655 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528cc08-9c15-4c27-b2ba-84a1484e9851 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:17.988939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528cc08-9c15-4c27-b2ba-84a1484e9851 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:17.989216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528cc08-9c15-4c27-b2ba-84a1484e9851 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:17.989922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528cc08-9c15-4c27-b2ba-84a1484e9851 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:17.990230 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528cc08-9c15-4c27-b2ba-84a1484e9851 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:17.991652 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528cc08-9c15-4c27-b2ba-84a1484e9851 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:17.996289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 437/893] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Jul 20 15:13:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:17.997572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6b3a8-6c75-4f03-8531-e1075fe682d5 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:17.997859 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6b3a8-6c75-4f03-8531-e1075fe682d5 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:17.998139 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6b3a8-6c75-4f03-8531-e1075fe682d5 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:17.999540 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 438/894] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:18.004946 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6b3a8-6c75-4f03-8531-e1075fe682d5 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:18.015216 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882a3d3-cc0e-4b6d-b68d-34ced0571fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:18.015818 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882a3d3-cc0e-4b6d-b68d-34ced0571fbd None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:18.016104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882a3d3-cc0e-4b6d-b68d-34ced0571fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:18.016385 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882a3d3-cc0e-4b6d-b68d-34ced0571fbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:18.017210 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882a3d3-cc0e-4b6d-b68d-34ced0571fbd None admin] RBAC: Adding query filter params (role.id=e2d60ba4ec674cbb8710971ec3f9bdfe, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=4e75a82cc7e94702ba0080fc65d6eef3) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:18.017531 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882a3d3-cc0e-4b6d-b68d-34ced0571fbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:18.019554 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882a3d3-cc0e-4b6d-b68d-34ced0571fbd None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:18.030289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 457/895] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:13:17 2022] GET /identity/v3/role_assignments?user.id=4e75a82cc7e94702ba0080fc65d6eef3&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=e2d60ba4ec674cbb8710971ec3f9bdfe => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:18.030289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11144051-9af8-4e4e-a65f-33bdc8a10fd8 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:18.030289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11144051-9af8-4e4e-a65f-33bdc8a10fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:18.030289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11144051-9af8-4e4e-a65f-33bdc8a10fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:18.046171 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-11144051-9af8-4e4e-a65f-33bdc8a10fd8 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:18.047517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-11144051-9af8-4e4e-a65f-33bdc8a10fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:18.132202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 458/896] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:18.152261 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6b3a8-6c75-4f03-8531-e1075fe682d5 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:18.155699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11144051-9af8-4e4e-a65f-33bdc8a10fd8 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:18.204535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3311a8-9930-4ad0-b0b3-536405241b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:18.205209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3311a8-9930-4ad0-b0b3-536405241b10 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:18.205559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3311a8-9930-4ad0-b0b3-536405241b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:18.206362 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3311a8-9930-4ad0-b0b3-536405241b10 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:18.207090 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3311a8-9930-4ad0-b0b3-536405241b10 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:18.207402 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3311a8-9930-4ad0-b0b3-536405241b10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:18.207653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:18.208770 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3311a8-9930-4ad0-b0b3-536405241b10 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:18.213971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 439/897] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Jul 20 15:13:18 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:18.276892 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11144051-9af8-4e4e-a65f-33bdc8a10fd8 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:18.319781 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7d406-ae0a-4611-b0e6-e1b1b5bed39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:18.320901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7d406-ae0a-4611-b0e6-e1b1b5bed39b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:18.321372 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7d406-ae0a-4611-b0e6-e1b1b5bed39b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:18.321810 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7d406-ae0a-4611-b0e6-e1b1b5bed39b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:18.322717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7d406-ae0a-4611-b0e6-e1b1b5bed39b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:18.324328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7d406-ae0a-4611-b0e6-e1b1b5bed39b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:18.339109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-14c7d406-ae0a-4611-b0e6-e1b1b5bed39b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:13:18.339109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:18.339109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:18.339109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:18.339109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:18.339109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:18.339109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:18.339109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:18.339109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:13:18.339109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:18.339109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:18.339109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:18.339109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:18.339109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:18.339109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:18.339109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:18.339109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:18.346794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:18.346794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:18.346794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:18.346794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:18.346794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:18.346794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:18.346794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:18.346794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:18.346794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:18.346794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:18.346794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:18.346794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:18.346794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:18.346794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:18.346794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:18.346794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:18.347904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:18.347904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:18.347904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:18.347904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:18.347904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:18.347904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:18.347904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:18.347904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:18.347904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:18.347904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:18.347904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:18.347904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:18.347904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:13:18.347904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:18.347904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 459/898] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:13:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:18.350788 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3311a8-9930-4ad0-b0b3-536405241b10 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:18.419152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:18.459963 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3311a8-9930-4ad0-b0b3-536405241b10 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:18.495694 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a655b78-e9c1-4c1d-97be-124148f50904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:18.496368 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a655b78-e9c1-4c1d-97be-124148f50904 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:18.496657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a655b78-e9c1-4c1d-97be-124148f50904 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:18.496938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a655b78-e9c1-4c1d-97be-124148f50904 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:18.497650 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a655b78-e9c1-4c1d-97be-124148f50904 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:18.497961 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a655b78-e9c1-4c1d-97be-124148f50904 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:18.499357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a655b78-e9c1-4c1d-97be-124148f50904 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:18.505408 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 440/899] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:13:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:18.511241 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7d406-ae0a-4611-b0e6-e1b1b5bed39b None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:18.616948 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7d406-ae0a-4611-b0e6-e1b1b5bed39b None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:18.626282 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:18.652185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1763a-6ffb-46af-89e4-521d4202e8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:18.652914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1763a-6ffb-46af-89e4-521d4202e8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:18.653202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1763a-6ffb-46af-89e4-521d4202e8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:18.653534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1763a-6ffb-46af-89e4-521d4202e8e4 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:18.654332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1763a-6ffb-46af-89e4-521d4202e8e4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:18.654648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1763a-6ffb-46af-89e4-521d4202e8e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:18.656029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1763a-6ffb-46af-89e4-521d4202e8e4 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:18.662099 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 460/900] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:13:18 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:18.667965 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a655b78-e9c1-4c1d-97be-124148f50904 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:18.741177 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a4096-8525-4ca1-866a-b08225f28239 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:18.741462 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a4096-8525-4ca1-866a-b08225f28239 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:18.741742 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a4096-8525-4ca1-866a-b08225f28239 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:18.743394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 461/901] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:13:18.751343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27681ab8-af19-4e91-875c-3f487385732a None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:18.752031 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27681ab8-af19-4e91-875c-3f487385732a None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:18.752994 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27681ab8-af19-4e91-875c-3f487385732a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:18.771575 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-27681ab8-af19-4e91-875c-3f487385732a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:18.773066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-27681ab8-af19-4e91-875c-3f487385732a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:18.776524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a655b78-e9c1-4c1d-97be-124148f50904 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:18.801982 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:18.813250 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a3cff-4c8b-48f5-95ee-4a5a7f4e7067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:18.814225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a3cff-4c8b-48f5-95ee-4a5a7f4e7067 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:18.814406 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a3cff-4c8b-48f5-95ee-4a5a7f4e7067 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:18.814664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a3cff-4c8b-48f5-95ee-4a5a7f4e7067 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:18.816101 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a3cff-4c8b-48f5-95ee-4a5a7f4e7067 None admin] RBAC: Adding query filter params (interface=public, service_id=4736fd794a48412992ae1c95b309b4a7, region_id=RegionOne) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:18.816420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a3cff-4c8b-48f5-95ee-4a5a7f4e7067 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:18.817859 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a3cff-4c8b-48f5-95ee-4a5a7f4e7067 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:18.823543 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 441/902] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Jul 20 15:13:18 2022] GET /identity/v3/endpoints?service_id=4736fd794a48412992ae1c95b309b4a7&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:18.828640 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a3cff-4c8b-48f5-95ee-4a5a7f4e7067 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:18.866283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 462/903] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:13:18.883198 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27681ab8-af19-4e91-875c-3f487385732a None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:18.970300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a3cff-4c8b-48f5-95ee-4a5a7f4e7067 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:19.009737 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27681ab8-af19-4e91-875c-3f487385732a None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:19.011898 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:19.018020 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a5087-5316-458c-bd9b-47cf59d9e9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:19.019400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a5087-5316-458c-bd9b-47cf59d9e9de None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:19.019901 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a5087-5316-458c-bd9b-47cf59d9e9de None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:19.020420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a5087-5316-458c-bd9b-47cf59d9e9de None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:19.021687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a5087-5316-458c-bd9b-47cf59d9e9de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:19.023865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a5087-5316-458c-bd9b-47cf59d9e9de None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:19.037082 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 442/904] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Jul 20 15:13:18 2022] GET /identity/v3/services => generated 1268 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:13:19.060791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b03b8-733c-46ab-97af-2a4a8b356e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:19.060791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b03b8-733c-46ab-97af-2a4a8b356e21 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:19.060791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b03b8-733c-46ab-97af-2a4a8b356e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:19.060791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b03b8-733c-46ab-97af-2a4a8b356e21 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:19.061162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b03b8-733c-46ab-97af-2a4a8b356e21 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:19.061162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b03b8-733c-46ab-97af-2a4a8b356e21 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:19.068697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-179b03b8-733c-46ab-97af-2a4a8b356e21 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:13:19.068697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:19.068697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:19.068697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:19.068697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:19.068697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:19.068697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:19.068697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:19.068697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:13:19.068697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:19.068697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:19.068697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:19.068697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:19.068697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:19.068697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:19.068697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:19.070082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:19.070082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:19.070082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:19.070082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:19.070082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:19.070082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:19.070082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:19.070082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:19.070082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:19.070082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:19.070082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:19.070082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:19.070082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:19.070082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:19.070082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:19.070082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:19.071012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:19.071012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:19.071012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:19.071012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:19.071012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:19.071012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:19.071012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:19.071012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:19.071012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:19.071012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:19.071012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:19.071012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:19.071012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:19.071012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:13:19.071012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:19.071012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 463/905] 10.222.0.78 () {56 vars in 1132 bytes} [Wed Jul 20 15:13:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:19.079579 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a5087-5316-458c-bd9b-47cf59d9e9de None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:19.146643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:19.205324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a5087-5316-458c-bd9b-47cf59d9e9de None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:19.242324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7b3c9-4a91-4216-8b77-496692f65cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:19.242990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7b3c9-4a91-4216-8b77-496692f65cab None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:19.243271 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7b3c9-4a91-4216-8b77-496692f65cab None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:19.243542 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7b3c9-4a91-4216-8b77-496692f65cab None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:19.244241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c7b3c9-4a91-4216-8b77-496692f65cab None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:19.244544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c7b3c9-4a91-4216-8b77-496692f65cab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:19.246077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c7b3c9-4a91-4216-8b77-496692f65cab None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:19.251349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 443/906] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Jul 20 15:13:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:19.257671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b03b8-733c-46ab-97af-2a4a8b356e21 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:19.365814 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b03b8-733c-46ab-97af-2a4a8b356e21 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:19.402032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e9939-3a64-47d1-8935-35b4cf2d5f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:19.402663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e9939-3a64-47d1-8935-35b4cf2d5f90 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:19.402954 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e9939-3a64-47d1-8935-35b4cf2d5f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:19.403236 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e9939-3a64-47d1-8935-35b4cf2d5f90 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:19.403956 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e9939-3a64-47d1-8935-35b4cf2d5f90 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:19.404268 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e9939-3a64-47d1-8935-35b4cf2d5f90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:19.405644 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e9939-3a64-47d1-8935-35b4cf2d5f90 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:19.411651 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 464/907] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Jul 20 15:13:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:19.417469 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7b3c9-4a91-4216-8b77-496692f65cab None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:19.522634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7b3c9-4a91-4216-8b77-496692f65cab None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:19.539744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:19.562453 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d64896-3e18-409e-a729-cee530cc826d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:19.563454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d64896-3e18-409e-a729-cee530cc826d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:19.563975 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d64896-3e18-409e-a729-cee530cc826d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:19.564481 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d64896-3e18-409e-a729-cee530cc826d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:19.565522 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d64896-3e18-409e-a729-cee530cc826d None admin] RBAC: Adding query filter params (interface=public, service_id=a13762f3ff1c446ca501d15c18f3d15c, region_id=RegionOne) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:19.566053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d64896-3e18-409e-a729-cee530cc826d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:19.567656 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d64896-3e18-409e-a729-cee530cc826d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:19.573652 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 444/908] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Jul 20 15:13:19 2022] GET /identity/v3/endpoints?service_id=a13762f3ff1c446ca501d15c18f3d15c&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:19.579069 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e9939-3a64-47d1-8935-35b4cf2d5f90 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:19.610644 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:19.681561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e9939-3a64-47d1-8935-35b4cf2d5f90 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:19.714853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada44fb-5339-48d7-b649-f9a397c17e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:19.715584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada44fb-5339-48d7-b649-f9a397c17e0f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:19.715867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada44fb-5339-48d7-b649-f9a397c17e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:19.716195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada44fb-5339-48d7-b649-f9a397c17e0f None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:19.716888 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada44fb-5339-48d7-b649-f9a397c17e0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:19.718281 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada44fb-5339-48d7-b649-f9a397c17e0f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:19.725072 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 465/909] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Jul 20 15:13:19 2022] GET /identity/v3/services => generated 1268 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:13:20.228940 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2024002-7365-477b-bc3f-8f9b268b524f None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:20.229621 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2024002-7365-477b-bc3f-8f9b268b524f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:20.229621 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2024002-7365-477b-bc3f-8f9b268b524f None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:20.231126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 445/910] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:20.237271 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b605731-3c21-4085-9ab5-bd3b324d9315 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:20.237584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b605731-3c21-4085-9ab5-bd3b324d9315 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:20.237842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b605731-3c21-4085-9ab5-bd3b324d9315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:20.256281 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-1b605731-3c21-4085-9ab5-bd3b324d9315 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:20.257479 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1b605731-3c21-4085-9ab5-bd3b324d9315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:20.304273 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d701ae-42f8-4d3c-a4e5-795f0c3be61d None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:20.304557 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d701ae-42f8-4d3c-a4e5-795f0c3be61d None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:20.304837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d701ae-42f8-4d3c-a4e5-795f0c3be61d None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:20.306435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 446/911] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:20.311405 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f55a2-769b-41de-9996-c107c65efb67 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:20.311694 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f55a2-769b-41de-9996-c107c65efb67 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:20.311973 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f55a2-769b-41de-9996-c107c65efb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:20.329215 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 466/912] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:20.329409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-1d5f55a2-769b-41de-9996-c107c65efb67 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:20.338295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-1d5f55a2-769b-41de-9996-c107c65efb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:20.352532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b605731-3c21-4085-9ab5-bd3b324d9315 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:20.377758 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:20.422304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 447/913] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:13:20.439573 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f55a2-769b-41de-9996-c107c65efb67 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:20.469658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b605731-3c21-4085-9ab5-bd3b324d9315 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:20.510333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fef4da-ff37-4469-a969-bcff975f6471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:20.511158 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fef4da-ff37-4469-a969-bcff975f6471 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:20.511433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fef4da-ff37-4469-a969-bcff975f6471 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:20.511718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fef4da-ff37-4469-a969-bcff975f6471 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:20.523155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fef4da-ff37-4469-a969-bcff975f6471 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:13:20.531408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-93fef4da-ff37-4469-a969-bcff975f6471 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:13:20.531408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:20.531408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:20.531408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:20.531408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:20.531408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:20.531408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:20.531408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:20.531408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:20.531408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:20.531408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:20.531408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:20.531408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:13:20.531408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:13:20.531408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:13:20.531408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:13:20.532402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:13:20.532402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:13:20.532402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:13:20.532402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:13:20.532402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:20.532402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:20.532402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:20.532402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:20.532402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:20.532402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:20.532402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:20.532402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:20.532402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:20.532402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:20.532402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:20.532402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:20.538889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:20.538889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:20.538889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:20.538889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:20.538889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:13:20.538889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:13:20.538889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:13:20.538889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:13:20.538889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:13:20.538889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:13:20.538889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:13:20.538889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:20.538889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 467/914] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Jul 20 15:13:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:20.538889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fef4da-ff37-4469-a969-bcff975f6471 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:20.569931 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f55a2-769b-41de-9996-c107c65efb67 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:20.585266 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:20.619908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9a1cf-348c-4865-b0d1-78a74bf18a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:20.620505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9a1cf-348c-4865-b0d1-78a74bf18a8e None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:20.620787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9a1cf-348c-4865-b0d1-78a74bf18a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:20.621064 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9a1cf-348c-4865-b0d1-78a74bf18a8e None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:20.621805 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9a1cf-348c-4865-b0d1-78a74bf18a8e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:20.623217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9a1cf-348c-4865-b0d1-78a74bf18a8e None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:20.650228 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:20.666577 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fef4da-ff37-4469-a969-bcff975f6471 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:20.707234 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 448/915] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Jul 20 15:13:20 2022] POST /identity/v3/services => generated 241 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:13:20.708634 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e322d-af5f-408f-b041-c0bf3976c808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:20.709084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e322d-af5f-408f-b041-c0bf3976c808 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:20.709084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e322d-af5f-408f-b041-c0bf3976c808 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:20.709381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e322d-af5f-408f-b041-c0bf3976c808 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:20.710040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e322d-af5f-408f-b041-c0bf3976c808 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:20.710384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e322d-af5f-408f-b041-c0bf3976c808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:20.711940 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e322d-af5f-408f-b041-c0bf3976c808 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:20.723050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 468/916] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Jul 20 15:13:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:20.729016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9a1cf-348c-4865-b0d1-78a74bf18a8e None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:20.842937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9a1cf-348c-4865-b0d1-78a74bf18a8e None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:20.884596 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46471-2d23-4e66-9181-91a23ee05639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:20.885395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46471-2d23-4e66-9181-91a23ee05639 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:20.885732 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46471-2d23-4e66-9181-91a23ee05639 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:20.886016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46471-2d23-4e66-9181-91a23ee05639 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:20.899210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb46471-2d23-4e66-9181-91a23ee05639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:20.900691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb46471-2d23-4e66-9181-91a23ee05639 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:20.919735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-adb46471-2d23-4e66-9181-91a23ee05639 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:20.919735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:20.919735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:20.919735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:20.919735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:20.919735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:20.919735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:20.919735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:20.919735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:20.919735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:20.919735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:20.919735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:20.919735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:20.919735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:20.919735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:20.920682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:20.920682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:20.920682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:20.920682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:20.920682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:20.920682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:20.920682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:20.920682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:20.920682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:20.920682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:20.920682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:20.920682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:20.920682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:20.920682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:20.920682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:20.920682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:20.920682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:20.921592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:20.921592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:20.921592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:20.921592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:20.921592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 449/917] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:20.927797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e322d-af5f-408f-b041-c0bf3976c808 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:21.034493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e322d-af5f-408f-b041-c0bf3976c808 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:21.069705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d7d56-ea4c-4aba-ad1f-22c8f14405f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:21.070817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d7d56-ea4c-4aba-ad1f-22c8f14405f9 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:21.071328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d7d56-ea4c-4aba-ad1f-22c8f14405f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:21.071808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d7d56-ea4c-4aba-ad1f-22c8f14405f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:21.072741 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d7d56-ea4c-4aba-ad1f-22c8f14405f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:21.073267 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d7d56-ea4c-4aba-ad1f-22c8f14405f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:21.075048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d7d56-ea4c-4aba-ad1f-22c8f14405f9 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:21.088231 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 469/918] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:21.104047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46471-2d23-4e66-9181-91a23ee05639 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:21.130446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:21.213471 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46471-2d23-4e66-9181-91a23ee05639 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:21.216370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28cc46-d62f-46f4-b820-68308254b2ce None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:21.216659 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28cc46-d62f-46f4-b820-68308254b2ce None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:21.216936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28cc46-d62f-46f4-b820-68308254b2ce None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:21.222372 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 470/919] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:21.224054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be997563-253f-4dd1-a638-c5b6e22277ec None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:21.224339 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be997563-253f-4dd1-a638-c5b6e22277ec None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:21.224618 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be997563-253f-4dd1-a638-c5b6e22277ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:21.243016 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-be997563-253f-4dd1-a638-c5b6e22277ec None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:21.244504 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-be997563-253f-4dd1-a638-c5b6e22277ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:21.244996 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:21.254326 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac0aa7-999f-4bb3-9698-8e4807099297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:21.255524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac0aa7-999f-4bb3-9698-8e4807099297 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:21.255817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac0aa7-999f-4bb3-9698-8e4807099297 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:21.256099 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac0aa7-999f-4bb3-9698-8e4807099297 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:21.266999 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac0aa7-999f-4bb3-9698-8e4807099297 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:21.269312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac0aa7-999f-4bb3-9698-8e4807099297 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:21.288349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-86ac0aa7-999f-4bb3-9698-8e4807099297 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:13:21.288349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:21.288349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:21.288349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:21.288349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:21.288349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:21.288349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:21.288349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:21.288349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:21.288349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:21.288349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:21.288349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:21.288349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:13:21.288349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:13:21.288349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:13:21.288349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:21.289347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:21.290327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:21.290327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:21.290327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:13:21.290327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:13:21.290327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:13:21.290327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:13:21.290327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:13:21.290327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:13:21.290327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:13:21.290327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:21.290327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 450/920] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Jul 20 15:13:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:21.298398 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac0aa7-999f-4bb3-9698-8e4807099297 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:21.321601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 471/921] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:13:21.347629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be997563-253f-4dd1-a638-c5b6e22277ec None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:21.410496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:21.434112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac0aa7-999f-4bb3-9698-8e4807099297 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:21.462978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be997563-253f-4dd1-a638-c5b6e22277ec None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:21.473453 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec52d7-71eb-4950-beff-344b4e7c142d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:21.474204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec52d7-71eb-4950-beff-344b4e7c142d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:21.474338 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec52d7-71eb-4950-beff-344b4e7c142d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:21.474674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec52d7-71eb-4950-beff-344b4e7c142d None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:21.475515 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec52d7-71eb-4950-beff-344b4e7c142d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:21.475868 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec52d7-71eb-4950-beff-344b4e7c142d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:21.477290 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec52d7-71eb-4950-beff-344b4e7c142d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:21.501498 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 451/922] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Jul 20 15:13:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:21.503967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec52d7-71eb-4950-beff-344b4e7c142d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:21.507305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:21.514476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2a653-d3aa-412b-8050-07ad525b60f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:21.522004 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2a653-d3aa-412b-8050-07ad525b60f2 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:21.522004 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2a653-d3aa-412b-8050-07ad525b60f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:21.522004 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2a653-d3aa-412b-8050-07ad525b60f2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:21.522004 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2a653-d3aa-412b-8050-07ad525b60f2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:21.522004 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2a653-d3aa-412b-8050-07ad525b60f2 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:21.536693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-5ba2a653-d3aa-412b-8050-07ad525b60f2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:13:21.536693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:21.536693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:21.536693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:21.536693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:21.536693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:21.536693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:21.536693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:21.536693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:13:21.536693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:21.536693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:21.536693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:21.536693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:21.536693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:21.536693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:21.536693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:21.536693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:21.537917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:21.537917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:21.537917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:21.537917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:21.537917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:21.537917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:21.537917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:21.537917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:21.537917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:21.537917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:21.537917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:21.537917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:21.537917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:21.537917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:21.537917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:21.537917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:21.539006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:21.539006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:21.539006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:21.539006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:21.539006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:21.539006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:21.539006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:21.539006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:21.539006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:21.539006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:21.539006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:21.539006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:21.539006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 15:13:21.539006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:21.542290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 472/923] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:13:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:21.548248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2a653-d3aa-412b-8050-07ad525b60f2 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:21.623735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec52d7-71eb-4950-beff-344b4e7c142d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:21.660402 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2605f15-4682-468a-89dc-e4c43a329e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:21.661360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2605f15-4682-468a-89dc-e4c43a329e08 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:21.661731 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2605f15-4682-468a-89dc-e4c43a329e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:21.662074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2605f15-4682-468a-89dc-e4c43a329e08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:21.666683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2a653-d3aa-412b-8050-07ad525b60f2 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:21.670848 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:21.673680 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2605f15-4682-468a-89dc-e4c43a329e08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:21.675319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2605f15-4682-468a-89dc-e4c43a329e08 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:21.694098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-b2605f15-4682-468a-89dc-e4c43a329e08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:21.694098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:21.694098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:21.694098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:21.694098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:21.694098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:21.694098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:21.694098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:21.694098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:21.694098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:21.694098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:21.694098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:21.694098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:21.694098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:21.694098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:21.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:21.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:21.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:21.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:21.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:21.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:21.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:21.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:21.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:21.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:21.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:21.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:21.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:21.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:21.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:21.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:21.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:21.696041 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:21.696041 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:21.696041 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:21.696041 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:21.696041 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 452/924] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:21.702052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2605f15-4682-468a-89dc-e4c43a329e08 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:21.711067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1ead2-7b05-4e2b-8c61-67cc9b69fcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:21.712312 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1ead2-7b05-4e2b-8c61-67cc9b69fcfc None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:21.712751 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1ead2-7b05-4e2b-8c61-67cc9b69fcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:21.713170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1ead2-7b05-4e2b-8c61-67cc9b69fcfc None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:21.714076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1ead2-7b05-4e2b-8c61-67cc9b69fcfc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:21.714543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1ead2-7b05-4e2b-8c61-67cc9b69fcfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:21.716070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1ead2-7b05-4e2b-8c61-67cc9b69fcfc None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:21.722051 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 473/925] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:13:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:21.731376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1ead2-7b05-4e2b-8c61-67cc9b69fcfc None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:21.797576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:21.811822 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2605f15-4682-468a-89dc-e4c43a329e08 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:21.842930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1ead2-7b05-4e2b-8c61-67cc9b69fcfc None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:21.850239 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cd0f7-2130-4e9e-b6da-66d19f6221d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:21.850929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cd0f7-2130-4e9e-b6da-66d19f6221d8 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:21.851440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cd0f7-2130-4e9e-b6da-66d19f6221d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:21.851549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cd0f7-2130-4e9e-b6da-66d19f6221d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:21.852193 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9cd0f7-2130-4e9e-b6da-66d19f6221d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:21.852498 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9cd0f7-2130-4e9e-b6da-66d19f6221d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:21.853853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9cd0f7-2130-4e9e-b6da-66d19f6221d8 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:21.869713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 453/926] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:21.869713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3e01e-395a-48e5-81a3-9e0d5e364949 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:21.869713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3e01e-395a-48e5-81a3-9e0d5e364949 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:21.869713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3e01e-395a-48e5-81a3-9e0d5e364949 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:21.869713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 454/927] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:21.871312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3e01e-395a-48e5-81a3-9e0d5e364949 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:21.890014 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc835b74-efd4-4f11-bc91-0f8f75e1037f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:21.890948 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc835b74-efd4-4f11-bc91-0f8f75e1037f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:21.891394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc835b74-efd4-4f11-bc91-0f8f75e1037f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:21.891815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc835b74-efd4-4f11-bc91-0f8f75e1037f None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:21.892674 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc835b74-efd4-4f11-bc91-0f8f75e1037f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:21.893649 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc835b74-efd4-4f11-bc91-0f8f75e1037f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:21.895063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc835b74-efd4-4f11-bc91-0f8f75e1037f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:21.904696 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 474/928] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Jul 20 15:13:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:21.905960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928dfc7-f157-42aa-a5ea-7c11b4b41790 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:21.906249 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928dfc7-f157-42aa-a5ea-7c11b4b41790 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:21.906546 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928dfc7-f157-42aa-a5ea-7c11b4b41790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:21.925138 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-1928dfc7-f157-42aa-a5ea-7c11b4b41790 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:21.926734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1928dfc7-f157-42aa-a5ea-7c11b4b41790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:21.931578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:21.987141 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3e01e-395a-48e5-81a3-9e0d5e364949 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:22.000867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 475/929] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:22.001510 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928dfc7-f157-42aa-a5ea-7c11b4b41790 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:22.027298 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc74e9-d5e6-43da-9065-45e9e82606f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:22.028379 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc74e9-d5e6-43da-9065-45e9e82606f5 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:22.028857 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc74e9-d5e6-43da-9065-45e9e82606f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:22.029171 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc74e9-d5e6-43da-9065-45e9e82606f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:22.036181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc74e9-d5e6-43da-9065-45e9e82606f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:22.037634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc74e9-d5e6-43da-9065-45e9e82606f5 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:22.052502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-56cc74e9-d5e6-43da-9065-45e9e82606f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:22.052502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:22.052502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:22.052502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:22.052502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:22.052502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:22.052502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:22.052502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:22.052502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:22.052502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:22.052502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:22.052502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:22.052502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:13:22.052502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:13:22.052502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:13:22.053448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:13:22.053448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:22.053448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:22.053448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:22.053448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:22.053448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:22.053448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:22.053448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:22.053448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:22.053448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:22.053448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:22.053448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:22.053448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:22.053448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:22.053448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:22.053448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:22.053448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:22.054812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:13:22.054812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:13:22.054812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:13:22.054812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:13:22.054812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:13:22.054812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:13:22.054812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:22.054812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:22.054812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 455/930] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:22.054812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc74e9-d5e6-43da-9065-45e9e82606f5 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:22.068125 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:22.121244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928dfc7-f157-42aa-a5ea-7c11b4b41790 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:22.157647 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1b09c-9264-40b3-9b44-4f978f03c549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:22.158377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1b09c-9264-40b3-9b44-4f978f03c549 None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:22.158684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1b09c-9264-40b3-9b44-4f978f03c549 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:22.158939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1b09c-9264-40b3-9b44-4f978f03c549 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:22.159052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc74e9-d5e6-43da-9065-45e9e82606f5 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:22.159700 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1b09c-9264-40b3-9b44-4f978f03c549 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:22.161904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1b09c-9264-40b3-9b44-4f978f03c549 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:22.172067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:22.192780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49af04-8ec8-4e9e-90f6-bb72dfdb63aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:22.193584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49af04-8ec8-4e9e-90f6-bb72dfdb63aa None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:22.193870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49af04-8ec8-4e9e-90f6-bb72dfdb63aa None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:22.194144 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49af04-8ec8-4e9e-90f6-bb72dfdb63aa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:22.194829 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49af04-8ec8-4e9e-90f6-bb72dfdb63aa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:22.196181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49af04-8ec8-4e9e-90f6-bb72dfdb63aa None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:22.208628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-7e49af04-8ec8-4e9e-90f6-bb72dfdb63aa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:13:22.208628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:22.208628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:22.208628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:22.208628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:22.208628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:22.208628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:22.208628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:22.208628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:13:22.208628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:22.208628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:22.208628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:22.208628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:22.208628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:22.208628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:22.208628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:22.209815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:22.209815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:22.209815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:22.209815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:22.209815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:22.209815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:22.209815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:22.209815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:22.209815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:22.209815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:22.209815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:22.209815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:22.209815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:22.209815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:22.209815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:22.209815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:22.220274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:22.220274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:22.220274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:22.220274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:22.220274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:22.220274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:22.220274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:22.220274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:22.220274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:22.220274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:22.220274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:22.220274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:22.220274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:22.220274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 15:13:22.220274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:22.220274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 456/931] 10.222.0.78 () {56 vars in 1132 bytes} [Wed Jul 20 15:13:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:22.221984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49af04-8ec8-4e9e-90f6-bb72dfdb63aa None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:22.279101 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 476/932] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Jul 20 15:13:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:13:22.282684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1b09c-9264-40b3-9b44-4f978f03c549 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:22.325321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49af04-8ec8-4e9e-90f6-bb72dfdb63aa None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:22.386202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fc5c3-2b27-4a32-9895-0f59055fa5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:22.386918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fc5c3-2b27-4a32-9895-0f59055fa5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:22.387244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fc5c3-2b27-4a32-9895-0f59055fa5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:22.387522 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fc5c3-2b27-4a32-9895-0f59055fa5d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:22.388331 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547fc5c3-2b27-4a32-9895-0f59055fa5d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:22.388639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547fc5c3-2b27-4a32-9895-0f59055fa5d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:22.390111 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547fc5c3-2b27-4a32-9895-0f59055fa5d8 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:22.410453 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 457/933] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:13:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:22.415316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fc5c3-2b27-4a32-9895-0f59055fa5d8 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:22.431240 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1b09c-9264-40b3-9b44-4f978f03c549 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:22.446596 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:22.463597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:22.483909 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a53461-fe72-4436-9829-ac98b0c22d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:22.484870 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a53461-fe72-4436-9829-ac98b0c22d6c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:22.485316 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a53461-fe72-4436-9829-ac98b0c22d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:22.485595 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a53461-fe72-4436-9829-ac98b0c22d6c None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:22.486344 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a53461-fe72-4436-9829-ac98b0c22d6c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:22.486736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a53461-fe72-4436-9829-ac98b0c22d6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:22.488112 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a53461-fe72-4436-9829-ac98b0c22d6c None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:22.497973 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 477/934] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Jul 20 15:13:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:22.500014 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a53461-fe72-4436-9829-ac98b0c22d6c None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:22.548592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fc5c3-2b27-4a32-9895-0f59055fa5d8 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:22.587253 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a28d83-8b4e-4e40-9591-f5f1e3434177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:22.588317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a28d83-8b4e-4e40-9591-f5f1e3434177 None admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:22.588843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a28d83-8b4e-4e40-9591-f5f1e3434177 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:22.589211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a28d83-8b4e-4e40-9591-f5f1e3434177 None admin] PATH_INFO: `/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/4e75a82cc7e94702ba0080fc65d6eef3/roles/e2d60ba4ec674cbb8710971ec3f9bdfe` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:22.591650 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a28d83-8b4e-4e40-9591-f5f1e3434177 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdae1cb71dd449b8a18062b7b52889e4, user_id=4e75a82cc7e94702ba0080fc65d6eef3, role_id=e2d60ba4ec674cbb8710971ec3f9bdfe)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:22.593237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a28d83-8b4e-4e40-9591-f5f1e3434177 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:22.620754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a53461-fe72-4436-9829-ac98b0c22d6c None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:22.656766 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96071c67-7258-4da1-846d-4979efe1bbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:22.657465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96071c67-7258-4da1-846d-4979efe1bbfb None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:22.657744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96071c67-7258-4da1-846d-4979efe1bbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:22.658024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96071c67-7258-4da1-846d-4979efe1bbfb None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:22.658784 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96071c67-7258-4da1-846d-4979efe1bbfb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:22.659093 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96071c67-7258-4da1-846d-4979efe1bbfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:22.660625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96071c67-7258-4da1-846d-4979efe1bbfb None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:22.670747 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 478/935] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Jul 20 15:13:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:22.672229 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96071c67-7258-4da1-846d-4979efe1bbfb None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:22.715673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:22.723169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 458/936] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:13:22 2022] PUT /identity/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/4e75a82cc7e94702ba0080fc65d6eef3/roles/e2d60ba4ec674cbb8710971ec3f9bdfe => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:13:22.791152 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96071c67-7258-4da1-846d-4979efe1bbfb None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:22.838847 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf10e37-3bb5-4ced-8b3d-c8c5efb04d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:22.840051 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf10e37-3bb5-4ced-8b3d-c8c5efb04d50 None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:22.840544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf10e37-3bb5-4ced-8b3d-c8c5efb04d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:22.841036 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf10e37-3bb5-4ced-8b3d-c8c5efb04d50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:22.842146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf10e37-3bb5-4ced-8b3d-c8c5efb04d50 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:22.843790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf10e37-3bb5-4ced-8b3d-c8c5efb04d50 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:22.904260 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38363c45-72d5-4022-9bf5-057cc89826d3 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:22.904622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38363c45-72d5-4022-9bf5-057cc89826d3 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:22.904827 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38363c45-72d5-4022-9bf5-057cc89826d3 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:22.906388 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 459/937] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:13:22.911307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394ecc0-02da-464b-98a1-a78c0a42027c None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:22.911598 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394ecc0-02da-464b-98a1-a78c0a42027c None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:22.911992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394ecc0-02da-464b-98a1-a78c0a42027c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:22.929833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-b394ecc0-02da-464b-98a1-a78c0a42027c None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:22.931091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b394ecc0-02da-464b-98a1-a78c0a42027c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:22.989197 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 479/938] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Jul 20 15:13:22 2022] POST /identity/v3/endpoints => generated 347 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:13:23.025229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 460/939] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:13:23.050460 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf10e37-3bb5-4ced-8b3d-c8c5efb04d50 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:23.162301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:23.170836 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf10e37-3bb5-4ced-8b3d-c8c5efb04d50 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:23.208527 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0de48-4ad7-45f7-be21-79f34eca95fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:23.209674 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0de48-4ad7-45f7-be21-79f34eca95fe None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:23.210144 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0de48-4ad7-45f7-be21-79f34eca95fe None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:23.210691 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0de48-4ad7-45f7-be21-79f34eca95fe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:23.211530 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0de48-4ad7-45f7-be21-79f34eca95fe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:23.213176 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0de48-4ad7-45f7-be21-79f34eca95fe None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:23.226421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-34a0de48-4ad7-45f7-be21-79f34eca95fe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:13:23.226421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:23.226421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:23.226421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:23.226421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:23.226421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:23.226421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:23.226421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:23.226421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:13:23.226421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:23.226421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:23.226421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:23.226421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:23.226421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:23.226421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:23.226421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:23.226421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:23.227518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:23.227518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:23.227518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:23.227518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:23.227518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:23.227518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:23.227518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:23.227518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:23.227518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:23.227518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:23.227518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:23.227518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:23.227518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:23.227518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:23.227518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:23.227518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:23.228394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:23.228394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:23.228394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:23.228394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:23.228394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:23.228394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:23.228394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:23.228394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:23.228394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:23.228394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:23.228394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:23.228394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:23.228394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:13:23.228394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:23.230962 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 480/940] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Jul 20 15:13:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:23.236790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394ecc0-02da-464b-98a1-a78c0a42027c None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:23.257143 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:23.343922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394ecc0-02da-464b-98a1-a78c0a42027c None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:23.380625 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d2e01-bcb2-4913-9a6b-5043ef9e6b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:23.381300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d2e01-bcb2-4913-9a6b-5043ef9e6b9e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:23.381586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d2e01-bcb2-4913-9a6b-5043ef9e6b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:23.381865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d2e01-bcb2-4913-9a6b-5043ef9e6b9e None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:23.382616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d2e01-bcb2-4913-9a6b-5043ef9e6b9e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:23.382925 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d2e01-bcb2-4913-9a6b-5043ef9e6b9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:23.384292 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d2e01-bcb2-4913-9a6b-5043ef9e6b9e None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:23.389680 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 461/941] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Jul 20 15:13:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:23.395176 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0de48-4ad7-45f7-be21-79f34eca95fe None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:23.507012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0de48-4ad7-45f7-be21-79f34eca95fe None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:23.545859 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe305789-7a14-4c19-b33e-971688c195d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:23.546630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe305789-7a14-4c19-b33e-971688c195d9 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:23.546922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe305789-7a14-4c19-b33e-971688c195d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:23.547204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe305789-7a14-4c19-b33e-971688c195d9 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:23.547983 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe305789-7a14-4c19-b33e-971688c195d9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:23.548299 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe305789-7a14-4c19-b33e-971688c195d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:23.549721 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe305789-7a14-4c19-b33e-971688c195d9 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:23.555323 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 481/942] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Jul 20 15:13:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:23.561124 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d2e01-bcb2-4913-9a6b-5043ef9e6b9e None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:23.646351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:23.664594 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d2e01-bcb2-4913-9a6b-5043ef9e6b9e None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:23.698535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b904e-54df-430b-b0ab-e8a443763e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:23.699200 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b904e-54df-430b-b0ab-e8a443763e7e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:23.699482 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b904e-54df-430b-b0ab-e8a443763e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:23.699759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b904e-54df-430b-b0ab-e8a443763e7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:23.700553 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b904e-54df-430b-b0ab-e8a443763e7e None admin] RBAC: Adding query filter params (interface=public, service_id=abd1289b702f49cc96caaf2ac9365c04, region_id=RegionOne) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:23.700868 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b904e-54df-430b-b0ab-e8a443763e7e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:23.702240 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b904e-54df-430b-b0ab-e8a443763e7e None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:23.708307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 462/943] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Jul 20 15:13:23 2022] GET /identity/v3/endpoints?service_id=abd1289b702f49cc96caaf2ac9365c04&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:23.713499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe305789-7a14-4c19-b33e-971688c195d9 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:23.739883 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:23.818018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe305789-7a14-4c19-b33e-971688c195d9 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:23.853944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23703e39-a9e3-4a15-9080-e44d4127d253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:23.854652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23703e39-a9e3-4a15-9080-e44d4127d253 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:23.854942 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23703e39-a9e3-4a15-9080-e44d4127d253 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:23.855223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23703e39-a9e3-4a15-9080-e44d4127d253 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:23.855910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23703e39-a9e3-4a15-9080-e44d4127d253 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:23.857291 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23703e39-a9e3-4a15-9080-e44d4127d253 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:23.864774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 482/944] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Jul 20 15:13:23 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:13:24.487930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc83c7-ee77-49af-98fe-98caf51bb708 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:24.488905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc83c7-ee77-49af-98fe-98caf51bb708 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:24.489426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc83c7-ee77-49af-98fe-98caf51bb708 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:24.491324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 463/945] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:24.496785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062f2b5-e4ce-4510-944e-8b60da1a27ca None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:24.497317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062f2b5-e4ce-4510-944e-8b60da1a27ca None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:24.497823 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062f2b5-e4ce-4510-944e-8b60da1a27ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:24.515326 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-b062f2b5-e4ce-4510-944e-8b60da1a27ca None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:24.516904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b062f2b5-e4ce-4510-944e-8b60da1a27ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:24.590413 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 483/946] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:24.610510 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc83c7-ee77-49af-98fe-98caf51bb708 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:24.717839 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc83c7-ee77-49af-98fe-98caf51bb708 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:24.753678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f384b-158e-497c-b223-f83eca4212e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:24.754287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f384b-158e-497c-b223-f83eca4212e2 None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:24.754576 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f384b-158e-497c-b223-f83eca4212e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:24.754852 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f384b-158e-497c-b223-f83eca4212e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:24.755577 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f384b-158e-497c-b223-f83eca4212e2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:24.757008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f384b-158e-497c-b223-f83eca4212e2 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:24.758392 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: WARNING keystone.server.flask.common [None req-7d8f384b-158e-497c-b223-f83eca4212e2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 15:13:24.815851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.sql.core [None req-7d8f384b-158e-497c-b223-f83eca4212e2 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 15:13:24.815851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 15:13:24.815851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [parameters: {'id': 'c94fbc630ea8420085a7f356ad90f499', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 15:13:24.815851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80509) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 15:13:24.834430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-7d8f384b-158e-497c-b223-f83eca4212e2 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c94fbc630ea8420085a7f356ad90f499.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c94fbc630ea8420085a7f356ad90f499. Jul 20 15:13:24.834430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:24.834430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 15:13:24.834430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 15:13:24.834430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 15:13:24.834430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 15:13:24.834430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 15:13:24.834430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 15:13:24.834430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 15:13:24.834430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application conn.query(q) Jul 20 15:13:24.834430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 15:13:24.834430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 15:13:24.834430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application result.read() Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 20 15:13:24.839890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 20 15:13:24.840885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 15:13:24.840885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application next(self.gen) Jul 20 15:13:24.840885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 15:13:24.840885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application yield resource Jul 20 15:13:24.840885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 15:13:24.840885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application next(self.gen) Jul 20 15:13:24.840885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 15:13:24.840885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 15:13:24.840885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 15:13:24.840885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 15:13:24.840885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 15:13:24.840885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise self.value Jul 20 15:13:24.840885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 15:13:24.840885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 15:13:24.840885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 15:13:24.840885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application session.commit() Jul 20 15:13:24.840885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 15:13:24.841809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 15:13:24.841809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 15:13:24.841809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 15:13:24.841809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 15:13:24.841809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self.session.flush() Jul 20 15:13:24.841809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 15:13:24.841809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 15:13:24.841809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 15:13:24.841809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 15:13:24.841809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 15:13:24.841809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application compat.raise_( Jul 20 15:13:24.841809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 15:13:24.841809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception Jul 20 15:13:24.841809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 15:13:24.841809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 15:13:24.841809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 15:13:24.841809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 15:13:24.842751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 15:13:24.842751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 15:13:24.842751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 15:13:24.842751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 15:13:24.842751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 15:13:24.842751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 15:13:24.842751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 15:13:24.842751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 15:13:24.842751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 15:13:24.842751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 15:13:24.842751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 15:13:24.842751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 15:13:24.842751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 15:13:24.842751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 15:13:24.842751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 15:13:24.843588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 15:13:24.843588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 15:13:24.843588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception Jul 20 15:13:24.843588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 15:13:24.843588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 15:13:24.843588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 15:13:24.843588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 15:13:24.843588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 15:13:24.843588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 15:13:24.843588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 15:13:24.843588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application conn.query(q) Jul 20 15:13:24.843588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 15:13:24.843588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 15:13:24.843588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 15:13:24.843588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application result.read() Jul 20 15:13:24.843588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 15:13:24.843588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 15:13:24.844476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 15:13:24.844476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 15:13:24.844476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 15:13:24.844476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 15:13:24.844476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 15:13:24.844476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 15:13:24.844476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 15:13:24.844476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 15:13:24.844476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [parameters: {'id': 'c94fbc630ea8420085a7f356ad90f499', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 15:13:24.844476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 15:13:24.844476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:24.844476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:24.844476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:24.844476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:24.844476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 20 15:13:24.845272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 20 15:13:24.845272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 15:13:24.845272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 15:13:24.845272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 20 15:13:24.845272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:24.845272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:24.845272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:24.845272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:24.845272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:24.845272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:24.845272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:24.845272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:24.845272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:24.845272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:24.845272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:24.845272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:24.845272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:24.846210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:24.846210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 20 15:13:24.846210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 20 15:13:24.846210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:24.846210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:24.846210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 20 15:13:24.846210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 20 15:13:24.846210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c94fbc630ea8420085a7f356ad90f499. Jul 20 15:13:24.846210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:24.846210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 464/947] 10.222.0.78 () {60 vars in 1140 bytes} [Wed Jul 20 15:13:24 2022] POST /identity/v3/projects => generated 262 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:24.853828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062f2b5-e4ce-4510-944e-8b60da1a27ca None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:24.863143 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6ab68-8431-4f54-b140-8b76cb381cf8 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:24.863456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6ab68-8431-4f54-b140-8b76cb381cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:24.863737 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6ab68-8431-4f54-b140-8b76cb381cf8 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:24.868260 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 465/948] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:13:24.873360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7da0d-e3ab-46e9-a26f-2be1f798f374 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:24.873651 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7da0d-e3ab-46e9-a26f-2be1f798f374 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:24.874099 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7da0d-e3ab-46e9-a26f-2be1f798f374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:24.880593 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:24.898633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:24.900449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-e0c7da0d-e3ab-46e9-a26f-2be1f798f374 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:24.902056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e0c7da0d-e3ab-46e9-a26f-2be1f798f374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:24.980310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062f2b5-e4ce-4510-944e-8b60da1a27ca None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:24.990690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 466/949] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:25.005085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7da0d-e3ab-46e9-a26f-2be1f798f374 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:25.017232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c2127-e9c8-40bf-86f4-0f778ee5070f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:25.018240 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c2127-e9c8-40bf-86f4-0f778ee5070f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:25.018560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c2127-e9c8-40bf-86f4-0f778ee5070f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:25.018845 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c2127-e9c8-40bf-86f4-0f778ee5070f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:25.028716 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c2127-e9c8-40bf-86f4-0f778ee5070f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:25.030197 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c2127-e9c8-40bf-86f4-0f778ee5070f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:25.049409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-692c2127-e9c8-40bf-86f4-0f778ee5070f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:25.049409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:25.049409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:25.049409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:25.049409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:25.049409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:25.049409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:25.049409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:25.049409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:25.049409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:25.049409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:25.049409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:25.049409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:13:25.049409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:13:25.049409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:13:25.050389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:13:25.050389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:25.050389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:25.050389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:25.050389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:25.050389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:25.050389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:25.050389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:25.050389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:25.050389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:25.050389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:25.050389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:25.050389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:25.050389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:25.050389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:25.050389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:25.050389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:25.051804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:13:25.051804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:13:25.051804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:13:25.051804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:13:25.051804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:13:25.051804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:13:25.051804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:25.051804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:25.051804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 484/950] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:25.058246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c2127-e9c8-40bf-86f4-0f778ee5070f None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:25.139694 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7da0d-e3ab-46e9-a26f-2be1f798f374 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:25.157391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:25.176832 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:25.182029 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46412d-6c42-4954-97ae-b93fe6d4fe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:25.183098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46412d-6c42-4954-97ae-b93fe6d4fe13 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:25.183369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46412d-6c42-4954-97ae-b93fe6d4fe13 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:25.183744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46412d-6c42-4954-97ae-b93fe6d4fe13 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:25.191923 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c2127-e9c8-40bf-86f4-0f778ee5070f None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:25.195548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46412d-6c42-4954-97ae-b93fe6d4fe13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:13:25.204743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-4b46412d-6c42-4954-97ae-b93fe6d4fe13 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:13:25.204743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:25.204743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:25.204743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:25.204743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:25.204743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:25.204743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:25.204743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:25.204743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:25.204743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:25.204743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:25.204743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:25.204743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:13:25.204743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:13:25.204743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:13:25.204743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:13:25.210204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:13:25.210204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:13:25.210204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:13:25.210204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:13:25.210204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:25.210204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:25.210204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:25.210204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:25.210204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:25.210204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:25.210204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:25.210204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:25.210204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:25.210204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:25.210204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:25.210204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:25.211185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:25.211185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:25.211185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:25.211185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:25.211185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:13:25.211185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:13:25.211185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:13:25.211185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:13:25.211185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:13:25.211185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:13:25.211185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:13:25.211185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:25.211185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 467/951] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Jul 20 15:13:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:25.211185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875b0b3-50ae-4a30-8bcb-75facae6f44e None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:25.212034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875b0b3-50ae-4a30-8bcb-75facae6f44e None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:25.212034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875b0b3-50ae-4a30-8bcb-75facae6f44e None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:25.212034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 468/952] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:25.215429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336221a0-eba6-4cd4-ad07-a24211c053ba None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:25.215839 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336221a0-eba6-4cd4-ad07-a24211c053ba None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:25.216124 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336221a0-eba6-4cd4-ad07-a24211c053ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:25.233324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b7c05-5674-47f2-92e4-91f299e476af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:25.233491 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-336221a0-eba6-4cd4-ad07-a24211c053ba None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:25.233984 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b7c05-5674-47f2-92e4-91f299e476af None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:25.234284 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b7c05-5674-47f2-92e4-91f299e476af None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:25.234566 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b7c05-5674-47f2-92e4-91f299e476af None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:25.234664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-336221a0-eba6-4cd4-ad07-a24211c053ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:25.235314 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b7c05-5674-47f2-92e4-91f299e476af None admin] RBAC: Adding query filter params (name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:25.235622 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b7c05-5674-47f2-92e4-91f299e476af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:25.237559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b7c05-5674-47f2-92e4-91f299e476af None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:25.255326 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 485/953] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Jul 20 15:13:25 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:25.255907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b7c05-5674-47f2-92e4-91f299e476af None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:25.331073 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 469/954] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:13:25.349027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336221a0-eba6-4cd4-ad07-a24211c053ba None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:25.382121 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b7c05-5674-47f2-92e4-91f299e476af None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:25.422899 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9e30f-804c-4bba-8351-da5f63180b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:25.423261 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9e30f-804c-4bba-8351-da5f63180b8f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:25.423549 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9e30f-804c-4bba-8351-da5f63180b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:25.423899 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9e30f-804c-4bba-8351-da5f63180b8f None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:25.425186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9e30f-804c-4bba-8351-da5f63180b8f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:25.425497 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9e30f-804c-4bba-8351-da5f63180b8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:25.426911 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9e30f-804c-4bba-8351-da5f63180b8f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:25.429455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:25.439184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 486/955] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Jul 20 15:13:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:25.448066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9e30f-804c-4bba-8351-da5f63180b8f None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:25.467705 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336221a0-eba6-4cd4-ad07-a24211c053ba None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:25.491782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:25.505192 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad458a-8cb5-4ad3-861b-e79353b23b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:25.506040 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad458a-8cb5-4ad3-861b-e79353b23b39 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:25.506314 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad458a-8cb5-4ad3-861b-e79353b23b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:25.506582 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad458a-8cb5-4ad3-861b-e79353b23b39 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:25.507216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad458a-8cb5-4ad3-861b-e79353b23b39 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:25.508700 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad458a-8cb5-4ad3-861b-e79353b23b39 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:25.524524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-8fad458a-8cb5-4ad3-861b-e79353b23b39 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:13:25.524524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:25.524524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:25.524524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:25.524524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:25.524524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:25.524524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:25.524524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:25.524524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:13:25.524524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:25.524524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:25.524524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:25.524524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:25.524524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:25.524524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:25.524524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:25.524524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:25.525888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:25.525888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:25.525888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:25.525888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:25.525888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:25.525888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:25.525888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:25.525888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:25.525888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:25.525888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:25.525888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:25.525888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:25.525888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:25.525888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:25.525888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:25.525888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:25.527098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:25.527098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:25.527098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:25.527098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:25.527098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:25.527098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:25.527098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:25.527098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:25.527098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:25.527098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:25.527098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:25.527098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:25.527098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:13:25.527098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:25.534361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 470/956] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:25.536939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad458a-8cb5-4ad3-861b-e79353b23b39 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:25.570008 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9e30f-804c-4bba-8351-da5f63180b8f None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:25.607560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb116335-2a5e-447a-9166-0ae7dffdd2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:25.608420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb116335-2a5e-447a-9166-0ae7dffdd2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:25.608754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb116335-2a5e-447a-9166-0ae7dffdd2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:25.609034 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb116335-2a5e-447a-9166-0ae7dffdd2c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:25.621206 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb116335-2a5e-447a-9166-0ae7dffdd2c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:25.622429 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb116335-2a5e-447a-9166-0ae7dffdd2c2 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:25.641538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-cb116335-2a5e-447a-9166-0ae7dffdd2c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:25.641538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:25.641538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:25.641538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:25.641538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:25.641538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:25.641538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:25.641538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:25.641538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:25.641538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:25.641538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:25.641538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:25.641538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:25.641538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:25.641538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:25.642734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:25.642734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:25.642734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:25.642734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:25.642734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:25.642734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:25.642734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:25.642734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:25.642734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:25.642734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:25.642734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:25.642734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:25.642734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:25.642734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:25.642734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:25.642734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:25.642734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:25.644352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:25.644352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:25.644352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:25.644352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:25.644352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 487/957] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:25.649259 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb116335-2a5e-447a-9166-0ae7dffdd2c2 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:25.651353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad458a-8cb5-4ad3-861b-e79353b23b39 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:25.687798 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343b130-c76a-4341-85a2-c3f3bf45ff88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:25.688709 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343b130-c76a-4341-85a2-c3f3bf45ff88 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:25.689164 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343b130-c76a-4341-85a2-c3f3bf45ff88 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:25.690152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343b130-c76a-4341-85a2-c3f3bf45ff88 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:25.691022 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343b130-c76a-4341-85a2-c3f3bf45ff88 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:25.691493 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343b130-c76a-4341-85a2-c3f3bf45ff88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:25.693107 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343b130-c76a-4341-85a2-c3f3bf45ff88 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:25.703454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 471/958] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Jul 20 15:13:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:25.704484 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343b130-c76a-4341-85a2-c3f3bf45ff88 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:25.706781 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:25.760115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb116335-2a5e-447a-9166-0ae7dffdd2c2 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:25.761848 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:25.802683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba533d-865e-4903-b5d5-e6c90a473d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:25.803463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba533d-865e-4903-b5d5-e6c90a473d34 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:25.803841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba533d-865e-4903-b5d5-e6c90a473d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:25.804212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba533d-865e-4903-b5d5-e6c90a473d34 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:25.805137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba533d-865e-4903-b5d5-e6c90a473d34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:25.805544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba533d-865e-4903-b5d5-e6c90a473d34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:25.807203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba533d-865e-4903-b5d5-e6c90a473d34 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:25.825152 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 488/959] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:25.826156 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba533d-865e-4903-b5d5-e6c90a473d34 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:25.827518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343b130-c76a-4341-85a2-c3f3bf45ff88 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:25.872002 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43291d81-1eb1-4127-a71b-8a14a525ac53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:25.872666 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43291d81-1eb1-4127-a71b-8a14a525ac53 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:25.872947 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43291d81-1eb1-4127-a71b-8a14a525ac53 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:25.873235 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43291d81-1eb1-4127-a71b-8a14a525ac53 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:25.873924 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43291d81-1eb1-4127-a71b-8a14a525ac53 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:25.874228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43291d81-1eb1-4127-a71b-8a14a525ac53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:25.875609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43291d81-1eb1-4127-a71b-8a14a525ac53 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:25.890962 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 472/960] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Jul 20 15:13:25 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:25.958948 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba533d-865e-4903-b5d5-e6c90a473d34 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:25.971730 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:25.997359 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928af0d-4ccf-474b-89a1-afc558572d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:25.998187 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928af0d-4ccf-474b-89a1-afc558572d59 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:25.998583 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928af0d-4ccf-474b-89a1-afc558572d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:25.998930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928af0d-4ccf-474b-89a1-afc558572d59 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:26.000331 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd92a19-8e8d-4965-a7db-a0ef6ca09d48 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:26.000625 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd92a19-8e8d-4965-a7db-a0ef6ca09d48 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:26.000898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd92a19-8e8d-4965-a7db-a0ef6ca09d48 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:26.005897 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 473/961] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:13:26.011878 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930feef-9f0c-4343-b33f-aa81cf1cc226 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:26.012143 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930feef-9f0c-4343-b33f-aa81cf1cc226 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:26.012346 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930feef-9f0c-4343-b33f-aa81cf1cc226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:26.012636 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928af0d-4ccf-474b-89a1-afc558572d59 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:26.014509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928af0d-4ccf-474b-89a1-afc558572d59 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:26.031130 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-7930feef-9f0c-4343-b33f-aa81cf1cc226 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:26.032342 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-7930feef-9f0c-4343-b33f-aa81cf1cc226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:26.032458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-2928af0d-4ccf-474b-89a1-afc558572d59 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:13:26.032458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:26.032458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:26.032458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:26.032458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:26.032458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:26.032458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:26.032458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:26.032458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:26.032458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:26.032458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:26.032458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:26.032458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:13:26.032458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:13:26.032458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:13:26.032458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:26.033941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:26.034960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:26.034960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:26.034960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:13:26.034960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:13:26.034960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:13:26.034960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:13:26.034960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:13:26.034960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:13:26.034960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 15:13:26.034960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:26.037691 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 489/962] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Jul 20 15:13:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:26.045805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928af0d-4ccf-474b-89a1-afc558572d59 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:26.154366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 474/963] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:13:26.168431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928af0d-4ccf-474b-89a1-afc558572d59 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:26.180284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930feef-9f0c-4343-b33f-aa81cf1cc226 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:26.204435 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40149e53-6e6b-4747-8a86-4ff14e9b4315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:26.205138 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40149e53-6e6b-4747-8a86-4ff14e9b4315 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:26.205422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40149e53-6e6b-4747-8a86-4ff14e9b4315 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:26.205700 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40149e53-6e6b-4747-8a86-4ff14e9b4315 None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:26.206571 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40149e53-6e6b-4747-8a86-4ff14e9b4315 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:26.206883 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40149e53-6e6b-4747-8a86-4ff14e9b4315 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:26.208318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40149e53-6e6b-4747-8a86-4ff14e9b4315 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:26.226699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 490/964] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Jul 20 15:13:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:26.232140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40149e53-6e6b-4747-8a86-4ff14e9b4315 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:26.234901 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:26.262136 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:26.306471 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930feef-9f0c-4343-b33f-aa81cf1cc226 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:26.340189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40149e53-6e6b-4747-8a86-4ff14e9b4315 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:26.341188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db9472-c816-479f-a13a-e7cf56f8093c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:26.341990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db9472-c816-479f-a13a-e7cf56f8093c None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:26.342290 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db9472-c816-479f-a13a-e7cf56f8093c None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:26.342623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db9472-c816-479f-a13a-e7cf56f8093c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:26.343275 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db9472-c816-479f-a13a-e7cf56f8093c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:26.344681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db9472-c816-479f-a13a-e7cf56f8093c None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:26.357653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-79db9472-c816-479f-a13a-e7cf56f8093c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:13:26.357653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:26.357653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:26.357653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:26.357653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:26.357653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:26.357653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:26.357653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:26.357653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:13:26.357653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:26.357653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:26.357653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:26.357653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:26.357653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:26.357653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:26.357653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:26.357653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:26.358738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:26.358738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:26.358738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:26.358738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:26.358738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:26.358738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:26.358738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:26.358738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:26.358738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:26.358738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:26.358738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:26.358738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:26.358738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:26.358738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:26.358738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:26.358738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:26.359657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:26.359657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:26.359657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:26.359657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:26.359657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:26.359657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:26.359657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:26.359657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:26.359657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:26.359657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:26.359657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:26.359657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:26.359657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:13:26.359657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:26.359657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 475/965] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Jul 20 15:13:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:26.365556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db9472-c816-479f-a13a-e7cf56f8093c None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:26.376168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0c5a3-19b4-483d-9ec4-f49781d1c34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:26.377155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0c5a3-19b4-483d-9ec4-f49781d1c34d None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:26.377438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0c5a3-19b4-483d-9ec4-f49781d1c34d None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:26.377715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0c5a3-19b4-483d-9ec4-f49781d1c34d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:26.448679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0c5a3-19b4-483d-9ec4-f49781d1c34d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:26.450136 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0c5a3-19b4-483d-9ec4-f49781d1c34d None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:26.479920 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:26.490834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db9472-c816-479f-a13a-e7cf56f8093c None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:26.501079 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:26.527677 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e6841-595f-4909-a498-dda3b9ac3e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:26.528567 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e6841-595f-4909-a498-dda3b9ac3e1c None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:26.528768 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e6841-595f-4909-a498-dda3b9ac3e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:26.529047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e6841-595f-4909-a498-dda3b9ac3e1c None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:26.529767 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e6841-595f-4909-a498-dda3b9ac3e1c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:26.530073 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e6841-595f-4909-a498-dda3b9ac3e1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:26.531510 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e6841-595f-4909-a498-dda3b9ac3e1c None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:26.542231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 476/966] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Jul 20 15:13:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:26.543468 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e6841-595f-4909-a498-dda3b9ac3e1c None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:26.596522 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 491/967] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:13:26 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 15:13:26.607235 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0c5a3-19b4-483d-9ec4-f49781d1c34d None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:26.667354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e6841-595f-4909-a498-dda3b9ac3e1c None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:26.703173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677ef3f-8f20-4cf0-ab49-6c58afcce4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:26.703990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677ef3f-8f20-4cf0-ab49-6c58afcce4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:26.704372 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677ef3f-8f20-4cf0-ab49-6c58afcce4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:26.704740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677ef3f-8f20-4cf0-ab49-6c58afcce4f6 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:26.706139 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677ef3f-8f20-4cf0-ab49-6c58afcce4f6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:26.706567 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677ef3f-8f20-4cf0-ab49-6c58afcce4f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:26.708015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677ef3f-8f20-4cf0-ab49-6c58afcce4f6 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:26.713969 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 477/968] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Jul 20 15:13:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:26.719048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677ef3f-8f20-4cf0-ab49-6c58afcce4f6 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:26.723872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0c5a3-19b4-483d-9ec4-f49781d1c34d None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:26.752259 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:26.756705 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:26.760704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee848e95-ba84-458f-ac1e-fe3e9ff0d0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:26.761549 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee848e95-ba84-458f-ac1e-fe3e9ff0d0fe None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:26.761838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee848e95-ba84-458f-ac1e-fe3e9ff0d0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:26.762116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee848e95-ba84-458f-ac1e-fe3e9ff0d0fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:26.773633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee848e95-ba84-458f-ac1e-fe3e9ff0d0fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:26.775103 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee848e95-ba84-458f-ac1e-fe3e9ff0d0fe None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:26.797134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-ee848e95-ba84-458f-ac1e-fe3e9ff0d0fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:26.797134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:26.797134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:26.797134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:26.797134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:26.797134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:26.797134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:26.797134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:26.797134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:26.797134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:26.797134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:26.797134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:26.797134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:26.797134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:26.797134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:26.798230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:26.798230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:26.798230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:26.798230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:26.798230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:26.798230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:26.798230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:26.798230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:26.798230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:26.798230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:26.798230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:26.798230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:26.798230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:26.798230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:26.798230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:26.798230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:26.798230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:26.799435 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:26.799435 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:26.799435 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:26.799435 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:26.799714 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 492/969] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:26.800636 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee848e95-ba84-458f-ac1e-fe3e9ff0d0fe None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:26.841588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677ef3f-8f20-4cf0-ab49-6c58afcce4f6 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:26.891528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04161f-c30b-4215-9a59-4563b83f451a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:26.892335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04161f-c30b-4215-9a59-4563b83f451a None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:26.892835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04161f-c30b-4215-9a59-4563b83f451a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:26.893421 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04161f-c30b-4215-9a59-4563b83f451a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:26.894062 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f04161f-c30b-4215-9a59-4563b83f451a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:26.895550 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f04161f-c30b-4215-9a59-4563b83f451a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:26.918204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee848e95-ba84-458f-ac1e-fe3e9ff0d0fe None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:26.953382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29a2ec-018c-4258-b63d-3c013d6dcf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:26.954056 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29a2ec-018c-4258-b63d-3c013d6dcf3b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:26.954369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29a2ec-018c-4258-b63d-3c013d6dcf3b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:26.954650 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29a2ec-018c-4258-b63d-3c013d6dcf3b None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:26.955457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29a2ec-018c-4258-b63d-3c013d6dcf3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:26.955770 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29a2ec-018c-4258-b63d-3c013d6dcf3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:26.957190 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29a2ec-018c-4258-b63d-3c013d6dcf3b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:26.974946 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 493/970] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:26.976022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29a2ec-018c-4258-b63d-3c013d6dcf3b None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:26.980475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 478/971] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Jul 20 15:13:26 2022] POST /identity/v3/endpoints => generated 349 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:13:27.037602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:27.117113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29a2ec-018c-4258-b63d-3c013d6dcf3b None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:27.161089 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78c680-5cac-4659-96eb-c6b6725c7901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:27.161884 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78c680-5cac-4659-96eb-c6b6725c7901 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:27.162162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78c680-5cac-4659-96eb-c6b6725c7901 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:27.162462 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78c680-5cac-4659-96eb-c6b6725c7901 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:27.169739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c78c680-5cac-4659-96eb-c6b6725c7901 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:27.171208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c78c680-5cac-4659-96eb-c6b6725c7901 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:27.185605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-7c78c680-5cac-4659-96eb-c6b6725c7901 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:27.185605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:27.185605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:27.185605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:27.185605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:27.185605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:27.185605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:27.185605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:27.185605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:27.185605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:27.185605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:27.185605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:27.185605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:13:27.185605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:13:27.185605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:13:27.186756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:13:27.186756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:27.186756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:27.186756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:27.186756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:27.186756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:27.186756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:27.186756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:27.186756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:27.186756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:27.186756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:27.186756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:27.186756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:27.186756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:27.186756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:27.186756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:27.186756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:27.187939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:13:27.187939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:13:27.187939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:13:27.187939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:13:27.187939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:13:27.187939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:13:27.187939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:27.187939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:27.187939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 494/972] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:27.192775 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04161f-c30b-4215-9a59-4563b83f451a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:27.289802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:27.299057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04161f-c30b-4215-9a59-4563b83f451a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:27.338468 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcffd23-8ae7-426a-8148-b40c2abfecb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:27.339139 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcffd23-8ae7-426a-8148-b40c2abfecb9 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:27.339425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcffd23-8ae7-426a-8148-b40c2abfecb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:27.339705 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcffd23-8ae7-426a-8148-b40c2abfecb9 None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:27.340550 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcffd23-8ae7-426a-8148-b40c2abfecb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:27.340866 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcffd23-8ae7-426a-8148-b40c2abfecb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:27.342307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcffd23-8ae7-426a-8148-b40c2abfecb9 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:27.355459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 479/973] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:13:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:27.361070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78c680-5cac-4659-96eb-c6b6725c7901 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:27.464578 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:27.471551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78c680-5cac-4659-96eb-c6b6725c7901 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:27.505837 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c44a37-b831-4a43-a002-4041d229eb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:27.506755 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c44a37-b831-4a43-a002-4041d229eb77 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:27.507172 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c44a37-b831-4a43-a002-4041d229eb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:27.507610 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c44a37-b831-4a43-a002-4041d229eb77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:27.508568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c44a37-b831-4a43-a002-4041d229eb77 None admin] RBAC: Adding query filter params (role.id=e2d60ba4ec674cbb8710971ec3f9bdfe, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=4e75a82cc7e94702ba0080fc65d6eef3) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:27.509208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c44a37-b831-4a43-a002-4041d229eb77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:27.510755 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c44a37-b831-4a43-a002-4041d229eb77 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:27.517147 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 495/974] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:13:27 2022] GET /identity/v3/role_assignments?user.id=4e75a82cc7e94702ba0080fc65d6eef3&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=e2d60ba4ec674cbb8710971ec3f9bdfe => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:27.536328 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307f87b-5aaa-417d-a7a6-e86a2c09db2a None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:27.536861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307f87b-5aaa-417d-a7a6-e86a2c09db2a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:27.537050 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307f87b-5aaa-417d-a7a6-e86a2c09db2a None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:27.538685 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 480/975] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:27.544590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b95e94-3f5c-4cdd-80e4-dcc48cb66ed0 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:27.545117 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b95e94-3f5c-4cdd-80e4-dcc48cb66ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:27.545552 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b95e94-3f5c-4cdd-80e4-dcc48cb66ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:27.564439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-b7b95e94-3f5c-4cdd-80e4-dcc48cb66ed0 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:27.565929 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b7b95e94-3f5c-4cdd-80e4-dcc48cb66ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:27.646372 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 496/976] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:27.671621 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307f87b-5aaa-417d-a7a6-e86a2c09db2a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:27.780460 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307f87b-5aaa-417d-a7a6-e86a2c09db2a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:27.814158 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76731f-f06d-481a-86c2-e148c7e6cee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:27.815414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76731f-f06d-481a-86c2-e148c7e6cee4 None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:27.815701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76731f-f06d-481a-86c2-e148c7e6cee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:27.815976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76731f-f06d-481a-86c2-e148c7e6cee4 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:27.816688 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d76731f-f06d-481a-86c2-e148c7e6cee4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:27.818116 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d76731f-f06d-481a-86c2-e148c7e6cee4 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:27.889477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.sql.core [None req-7d76731f-f06d-481a-86c2-e148c7e6cee4 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 15:13:27.889477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 15:13:27.889477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [parameters: {'user_id': '1f0ddad2bae849b9843d3f6e131205ca', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 15:13:27.889477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80509) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 15:13:27.898156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-7d76731f-f06d-481a-86c2-e148c7e6cee4 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 15:13:27.898156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:27.898156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 15:13:27.898156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 15:13:27.898156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 15:13:27.898156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 15:13:27.898156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 15:13:27.898156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 15:13:27.898156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 15:13:27.898156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application conn.query(q) Jul 20 15:13:27.898156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 15:13:27.898156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 15:13:27.898156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 15:13:27.898156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application result.read() Jul 20 15:13:27.898156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 15:13:27.899088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application next(self.gen) Jul 20 15:13:27.900087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 15:13:27.900087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application yield resource Jul 20 15:13:27.900087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 15:13:27.900087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application next(self.gen) Jul 20 15:13:27.900087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 15:13:27.900087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 15:13:27.900087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 15:13:27.900087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 15:13:27.900087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 15:13:27.900087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise self.value Jul 20 15:13:27.900087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 15:13:27.900087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 15:13:27.900087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 15:13:27.900087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application session.commit() Jul 20 15:13:27.900087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 15:13:27.900087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 15:13:27.900087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 15:13:27.901033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 15:13:27.901033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 15:13:27.901033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self.session.flush() Jul 20 15:13:27.901033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 15:13:27.901033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 15:13:27.901033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 15:13:27.901033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 15:13:27.901033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 15:13:27.901033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application compat.raise_( Jul 20 15:13:27.901033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 15:13:27.901033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception Jul 20 15:13:27.901033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 15:13:27.901033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 15:13:27.901033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 15:13:27.901033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 15:13:27.901033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 15:13:27.901033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 15:13:27.902045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 15:13:27.902045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 15:13:27.902045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 20 15:13:27.902045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 20 15:13:27.902045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 15:13:27.902045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 15:13:27.902045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 15:13:27.902045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 15:13:27.902045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 15:13:27.902045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 15:13:27.902045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 15:13:27.902045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 15:13:27.902045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 15:13:27.902045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 15:13:27.902045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 15:13:27.902045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception Jul 20 15:13:27.902970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 15:13:27.902970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 15:13:27.902970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 15:13:27.902970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 15:13:27.902970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 15:13:27.902970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 15:13:27.902970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 15:13:27.902970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application conn.query(q) Jul 20 15:13:27.902970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 15:13:27.902970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 15:13:27.902970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 15:13:27.902970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application result.read() Jul 20 15:13:27.902970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 15:13:27.902970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 15:13:27.902970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 15:13:27.902970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 15:13:27.903853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 15:13:27.903853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 15:13:27.903853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 15:13:27.903853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 15:13:27.903853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 15:13:27.903853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 15:13:27.903853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [parameters: {'user_id': '1f0ddad2bae849b9843d3f6e131205ca', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 15:13:27.903853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 15:13:27.903853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:27.903853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:27.903853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:27.903853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:27.903853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:27.903853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:27.903853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:27.904787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:27.904787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:27.904787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:27.904787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:27.904787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:27.904787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:27.904787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:27.904787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 20 15:13:27.904787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 20 15:13:27.904787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:27.904787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:27.904787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:13:27.904787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:27.904787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:13:27.904787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:27.904787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 20 15:13:27.904787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 20 15:13:27.905799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 20 15:13:27.905799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 20 15:13:27.905799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 15:13:27.905799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 15:13:27.905799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 15:13:27.905799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:27.905799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 481/977] 10.222.0.78 () {60 vars in 1131 bytes} [Wed Jul 20 15:13:27 2022] POST /identity/v3/users => generated 160 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:27.915449 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b95e94-3f5c-4cdd-80e4-dcc48cb66ed0 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:28.007460 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:28.017032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b95e94-3f5c-4cdd-80e4-dcc48cb66ed0 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:28.052423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3abbe-0018-4346-b82d-8ea50bc81daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:28.053572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3abbe-0018-4346-b82d-8ea50bc81daf None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:28.054080 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3abbe-0018-4346-b82d-8ea50bc81daf None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:28.054581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3abbe-0018-4346-b82d-8ea50bc81daf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:28.062239 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3abbe-0018-4346-b82d-8ea50bc81daf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:28.063972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3abbe-0018-4346-b82d-8ea50bc81daf None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:28.079414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-1cd3abbe-0018-4346-b82d-8ea50bc81daf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 15:13:28.079414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:28.079414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:28.079414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:28.079414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:28.079414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:28.079414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:28.079414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:28.079414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:28.079414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:28.079414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:28.079414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:28.079414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:13:28.079414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:13:28.079414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:13:28.079414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:28.080493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:28.081568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:28.081568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:28.081568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:13:28.081568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:13:28.081568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:13:28.081568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:13:28.081568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:13:28.081568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:13:28.081568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 15:13:28.081568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:28.084242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 497/978] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:13:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:28.098662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76731f-f06d-481a-86c2-e148c7e6cee4 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:28.105750 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b67525-37b0-4955-9c1d-800faf0da8b7 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:28.106041 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b67525-37b0-4955-9c1d-800faf0da8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:28.106356 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b67525-37b0-4955-9c1d-800faf0da8b7 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:28.107999 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 498/979] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:13:28.113394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143218f-e5d7-4cb5-bec6-9efe3ae44d1a None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:28.113683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143218f-e5d7-4cb5-bec6-9efe3ae44d1a None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:28.113965 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143218f-e5d7-4cb5-bec6-9efe3ae44d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:28.118903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:28.131687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-3143218f-e5d7-4cb5-bec6-9efe3ae44d1a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:28.132936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-3143218f-e5d7-4cb5-bec6-9efe3ae44d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:28.221776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76731f-f06d-481a-86c2-e148c7e6cee4 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:28.238852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 499/980] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:13:28.265378 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143218f-e5d7-4cb5-bec6-9efe3ae44d1a None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:28.269563 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126efe7e-0a39-4122-ae0b-723a5d1d3f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:28.270445 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126efe7e-0a39-4122-ae0b-723a5d1d3f72 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:28.270904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126efe7e-0a39-4122-ae0b-723a5d1d3f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:28.271328 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126efe7e-0a39-4122-ae0b-723a5d1d3f72 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:28.272298 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126efe7e-0a39-4122-ae0b-723a5d1d3f72 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:28.272756 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126efe7e-0a39-4122-ae0b-723a5d1d3f72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:28.275180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126efe7e-0a39-4122-ae0b-723a5d1d3f72 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:28.326085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 482/981] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Jul 20 15:13:28 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:28.361302 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:28.383620 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143218f-e5d7-4cb5-bec6-9efe3ae44d1a None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:28.427006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad46592-438c-476a-bcd0-562def31e6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:28.427859 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad46592-438c-476a-bcd0-562def31e6ad None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:28.428246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad46592-438c-476a-bcd0-562def31e6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:28.428628 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad46592-438c-476a-bcd0-562def31e6ad None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:28.429498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad46592-438c-476a-bcd0-562def31e6ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:28.431029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad46592-438c-476a-bcd0-562def31e6ad None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:28.507165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 500/982] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Jul 20 15:13:28 2022] POST /identity/v3/services => generated 242 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:13:29.189039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd5e15-f751-46d8-85f1-513891100fe4 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:29.189586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd5e15-f751-46d8-85f1-513891100fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:29.189674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd5e15-f751-46d8-85f1-513891100fe4 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:29.191180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 483/983] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:29.196351 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834fba8-d3cf-4470-b580-6d1fcc3d2175 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:29.196825 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834fba8-d3cf-4470-b580-6d1fcc3d2175 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:29.196906 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834fba8-d3cf-4470-b580-6d1fcc3d2175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:29.213964 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-8834fba8-d3cf-4470-b580-6d1fcc3d2175 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:29.215183 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8834fba8-d3cf-4470-b580-6d1fcc3d2175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:29.289192 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 501/984] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:29.312027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd5e15-f751-46d8-85f1-513891100fe4 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:29.388126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:29.418627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd5e15-f751-46d8-85f1-513891100fe4 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:29.452638 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ddc48-d9fd-4de6-a54e-02d4dc413d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:29.453436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ddc48-d9fd-4de6-a54e-02d4dc413d3f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:29.453718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ddc48-d9fd-4de6-a54e-02d4dc413d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:29.453992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ddc48-d9fd-4de6-a54e-02d4dc413d3f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:29.454664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ddc48-d9fd-4de6-a54e-02d4dc413d3f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:29.456057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ddc48-d9fd-4de6-a54e-02d4dc413d3f None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:29.468202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-ce8ddc48-d9fd-4de6-a54e-02d4dc413d3f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:13:29.468202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:29.468202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:29.468202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:29.468202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:29.468202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:29.468202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:29.468202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:29.468202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:13:29.468202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:29.468202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:29.468202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:29.468202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:29.468202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:29.468202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:29.468202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:29.468202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:29.469216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:29.469216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:29.469216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:29.469216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:29.469216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:29.469216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:29.469216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:29.469216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:29.469216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:29.469216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:29.469216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:29.469216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:29.469216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:29.469216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:29.469216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:29.469216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:29.470104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:29.470104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:29.470104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:29.470104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:29.470104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:29.470104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:29.470104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:29.470104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:29.470104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:29.470104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:29.470104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:29.470104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:29.470104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:13:29.470104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:29.470104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 484/985] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Jul 20 15:13:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:29.477448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834fba8-d3cf-4470-b580-6d1fcc3d2175 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:29.543890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:29.578589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834fba8-d3cf-4470-b580-6d1fcc3d2175 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:29.613367 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f140ea9-32ec-48d6-a591-630ab8289c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:29.614100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f140ea9-32ec-48d6-a591-630ab8289c24 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:29.614408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f140ea9-32ec-48d6-a591-630ab8289c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:29.614684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f140ea9-32ec-48d6-a591-630ab8289c24 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:29.615412 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f140ea9-32ec-48d6-a591-630ab8289c24 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:29.615720 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f140ea9-32ec-48d6-a591-630ab8289c24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:29.617134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f140ea9-32ec-48d6-a591-630ab8289c24 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:29.627823 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 502/986] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Jul 20 15:13:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:29.629257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ddc48-d9fd-4de6-a54e-02d4dc413d3f None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:29.651323 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627c80a-67c3-4010-93a4-0e110813899a None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:29.651607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627c80a-67c3-4010-93a4-0e110813899a None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:29.651797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627c80a-67c3-4010-93a4-0e110813899a None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:29.653304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 503/987] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:13:29.658027 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b11f2-1f49-470e-ba9f-dc40594edd84 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:29.658887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b11f2-1f49-470e-ba9f-dc40594edd84 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:29.659172 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b11f2-1f49-470e-ba9f-dc40594edd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:29.677913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-d71b11f2-1f49-470e-ba9f-dc40594edd84 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:29.679300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d71b11f2-1f49-470e-ba9f-dc40594edd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:29.738309 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ddc48-d9fd-4de6-a54e-02d4dc413d3f None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:29.750618 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 504/988] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:13:29.773101 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b11f2-1f49-470e-ba9f-dc40594edd84 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:29.775783 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af98062-0284-479e-bdc8-7a2fe2cadb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:29.776376 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af98062-0284-479e-bdc8-7a2fe2cadb8f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:29.776658 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af98062-0284-479e-bdc8-7a2fe2cadb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:29.776931 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af98062-0284-479e-bdc8-7a2fe2cadb8f None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:29.777630 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af98062-0284-479e-bdc8-7a2fe2cadb8f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:29.777938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af98062-0284-479e-bdc8-7a2fe2cadb8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:29.779342 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af98062-0284-479e-bdc8-7a2fe2cadb8f None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:29.781467 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:29.786299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 485/989] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Jul 20 15:13:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:29.791356 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af98062-0284-479e-bdc8-7a2fe2cadb8f None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:29.839967 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:29.883005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b11f2-1f49-470e-ba9f-dc40594edd84 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:29.902733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af98062-0284-479e-bdc8-7a2fe2cadb8f None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:29.920123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78312858-1c13-42ff-8095-0b2063f81d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:29.921282 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78312858-1c13-42ff-8095-0b2063f81d38 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:29.921564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78312858-1c13-42ff-8095-0b2063f81d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:29.921838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78312858-1c13-42ff-8095-0b2063f81d38 None admin] PATH_INFO: `/v3/services/image` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:29.922528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78312858-1c13-42ff-8095-0b2063f81d38 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:29.924738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78312858-1c13-42ff-8095-0b2063f81d38 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:29.937012 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeefbea-71ca-48db-89f8-4fbae307d0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:29.937664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeefbea-71ca-48db-89f8-4fbae307d0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:29.937949 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeefbea-71ca-48db-89f8-4fbae307d0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:29.938226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeefbea-71ca-48db-89f8-4fbae307d0b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:29.938890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-78312858-1c13-42ff-8095-0b2063f81d38 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:13:29.938890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:29.938890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:29.938890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:29.938890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:29.938890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:29.938890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:29.938890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:29.938890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:13:29.938890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:29.938890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:29.938890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:29.938890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:29.938890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:29.938890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:29.938890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:29.938890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:29.940381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:29.940381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:29.940381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:29.940381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:29.940381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:29.940381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:29.940381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:29.940381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:29.940381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:29.940381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:29.940381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:29.940381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:29.940381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:29.940381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:29.940381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:29.940381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:29.941828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:29.941828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:29.941828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:29.941828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:29.941828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:29.941828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:29.941828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:29.941828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:29.941828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:29.941828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:29.941828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:29.941828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:29.941828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:13:29.941828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:29.942640 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeefbea-71ca-48db-89f8-4fbae307d0b9 None admin] RBAC: Adding query filter params (interface=admin, service_id=abd1289b702f49cc96caaf2ac9365c04, region_id=RegionOne) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:29.942640 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeefbea-71ca-48db-89f8-4fbae307d0b9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:29.942782 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 505/990] 10.222.0.78 () {56 vars in 1105 bytes} [Wed Jul 20 15:13:29 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:29.942853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeefbea-71ca-48db-89f8-4fbae307d0b9 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:29.946456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78312858-1c13-42ff-8095-0b2063f81d38 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:29.955753 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 486/991] 10.222.0.78 () {56 vars in 1249 bytes} [Wed Jul 20 15:13:29 2022] GET /identity/v3/endpoints?service_id=abd1289b702f49cc96caaf2ac9365c04&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:29.956914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeefbea-71ca-48db-89f8-4fbae307d0b9 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:30.025422 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:30.049849 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78312858-1c13-42ff-8095-0b2063f81d38 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:30.067017 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeefbea-71ca-48db-89f8-4fbae307d0b9 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:30.075040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:30.087006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af496d-b1c2-4b03-8872-4464478410d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:30.088582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af496d-b1c2-4b03-8872-4464478410d5 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:30.088582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af496d-b1c2-4b03-8872-4464478410d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:30.088582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af496d-b1c2-4b03-8872-4464478410d5 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:30.089068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af496d-b1c2-4b03-8872-4464478410d5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:30.089378 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af496d-b1c2-4b03-8872-4464478410d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:30.090816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af496d-b1c2-4b03-8872-4464478410d5 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:30.100936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 506/992] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:30.102422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af496d-b1c2-4b03-8872-4464478410d5 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:30.109393 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04b186-e08a-4d83-b4ca-86e2205c90f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:30.110066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04b186-e08a-4d83-b4ca-86e2205c90f8 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:30.110389 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04b186-e08a-4d83-b4ca-86e2205c90f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:30.110668 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04b186-e08a-4d83-b4ca-86e2205c90f8 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:30.111952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04b186-e08a-4d83-b4ca-86e2205c90f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:30.113319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04b186-e08a-4d83-b4ca-86e2205c90f8 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:30.120191 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 487/993] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Jul 20 15:13:29 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:13:30.223357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af496d-b1c2-4b03-8872-4464478410d5 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:30.259776 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28ff98-f37a-4c2d-be13-98177f168873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:30.260468 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28ff98-f37a-4c2d-be13-98177f168873 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:30.260749 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28ff98-f37a-4c2d-be13-98177f168873 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:30.261026 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28ff98-f37a-4c2d-be13-98177f168873 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:30.261744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28ff98-f37a-4c2d-be13-98177f168873 None admin] RBAC: Adding query filter params (type=image) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:30.262052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28ff98-f37a-4c2d-be13-98177f168873 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:30.263478 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28ff98-f37a-4c2d-be13-98177f168873 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:30.269709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 507/994] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:30 2022] GET /identity/v3/services?type=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:30.559884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f5dd0-2321-45cb-aaf4-eef1e9ef5e91 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:30.560402 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f5dd0-2321-45cb-aaf4-eef1e9ef5e91 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:30.560512 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f5dd0-2321-45cb-aaf4-eef1e9ef5e91 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:30.561950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 488/995] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:30.567187 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f9ee7-d0cb-46b6-925a-b72a039ade9e None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:30.567479 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f9ee7-d0cb-46b6-925a-b72a039ade9e None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:30.567756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f9ee7-d0cb-46b6-925a-b72a039ade9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:30.585106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-458f9ee7-d0cb-46b6-925a-b72a039ade9e None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:30.586307 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-458f9ee7-d0cb-46b6-925a-b72a039ade9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:30.634739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:30.654561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 508/996] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:30 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:30.677027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f5dd0-2321-45cb-aaf4-eef1e9ef5e91 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:30.783689 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f5dd0-2321-45cb-aaf4-eef1e9ef5e91 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:30.809330 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:30.820092 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd921951-fef3-48b7-a4e3-32cc11b834e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:30.820092 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd921951-fef3-48b7-a4e3-32cc11b834e5 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:30.820092 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd921951-fef3-48b7-a4e3-32cc11b834e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:30.820092 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd921951-fef3-48b7-a4e3-32cc11b834e5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:30.820436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd921951-fef3-48b7-a4e3-32cc11b834e5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:30.821102 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd921951-fef3-48b7-a4e3-32cc11b834e5 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:30.833264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-bd921951-fef3-48b7-a4e3-32cc11b834e5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:13:30.833264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:30.833264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:30.833264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:30.833264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:30.833264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:30.833264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:30.833264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:30.833264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:13:30.833264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:30.833264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:30.833264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:30.833264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:30.833264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:30.833264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:30.833264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:30.833264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:30.834609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:30.834609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:30.834609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:30.834609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:30.834609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:30.834609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:30.834609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:30.834609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:30.834609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:30.834609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:30.834609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:30.834609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:30.834609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:30.834609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:30.834609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:30.834609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:30.835552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:30.835552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:30.835552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:30.835552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:30.835552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:30.835552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:30.835552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:30.835552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:30.835552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:30.835552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:30.835552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:30.835552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:30.835552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:13:30.835552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:30.835552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 489/997] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:30.842876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f9ee7-d0cb-46b6-925a-b72a039ade9e None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:30.947812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f9ee7-d0cb-46b6-925a-b72a039ade9e None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:30.981784 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8b8d1-a324-4f9b-9673-b3eb4a1fbc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:30.982620 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8b8d1-a324-4f9b-9673-b3eb4a1fbc46 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:30.983001 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8b8d1-a324-4f9b-9673-b3eb4a1fbc46 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:30.983377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8b8d1-a324-4f9b-9673-b3eb4a1fbc46 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:30.984189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8b8d1-a324-4f9b-9673-b3eb4a1fbc46 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:30.984612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8b8d1-a324-4f9b-9673-b3eb4a1fbc46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:30.986189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8b8d1-a324-4f9b-9673-b3eb4a1fbc46 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:30.996376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 509/998] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Jul 20 15:13:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:30.997569 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd921951-fef3-48b7-a4e3-32cc11b834e5 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:31.100810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd921951-fef3-48b7-a4e3-32cc11b834e5 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:31.135402 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f0b82-6269-4077-bdbf-ee003bb89dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:31.136067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f0b82-6269-4077-bdbf-ee003bb89dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:31.136352 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f0b82-6269-4077-bdbf-ee003bb89dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:31.136630 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f0b82-6269-4077-bdbf-ee003bb89dc3 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:31.137338 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f0b82-6269-4077-bdbf-ee003bb89dc3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:31.137647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f0b82-6269-4077-bdbf-ee003bb89dc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:31.139051 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f0b82-6269-4077-bdbf-ee003bb89dc3 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:31.144609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 490/999] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Jul 20 15:13:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:31.150310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8b8d1-a324-4f9b-9673-b3eb4a1fbc46 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:31.198630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:31.253490 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8b8d1-a324-4f9b-9673-b3eb4a1fbc46 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:31.291468 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb02b7-90a4-4b2a-9fd0-1a97f57fb20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:31.292459 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb02b7-90a4-4b2a-9fd0-1a97f57fb20e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:31.292933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb02b7-90a4-4b2a-9fd0-1a97f57fb20e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:31.293394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb02b7-90a4-4b2a-9fd0-1a97f57fb20e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:31.294409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb02b7-90a4-4b2a-9fd0-1a97f57fb20e None admin] RBAC: Adding query filter params (interface=public, service_id=06a96a1ca0954afe93a7003bd056c13e, region_id=RegionOne) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:31.294934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb02b7-90a4-4b2a-9fd0-1a97f57fb20e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:31.296485 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb02b7-90a4-4b2a-9fd0-1a97f57fb20e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:31.304277 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 510/1000] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Jul 20 15:13:31 2022] GET /identity/v3/endpoints?service_id=06a96a1ca0954afe93a7003bd056c13e&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:31.310011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f0b82-6269-4077-bdbf-ee003bb89dc3 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:31.370920 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:31.417093 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f0b82-6269-4077-bdbf-ee003bb89dc3 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:31.450615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffecdcd-11d3-458c-97ec-1f7f8d399842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:31.451279 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffecdcd-11d3-458c-97ec-1f7f8d399842 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:31.451565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffecdcd-11d3-458c-97ec-1f7f8d399842 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:31.451845 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffecdcd-11d3-458c-97ec-1f7f8d399842 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:31.452528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffecdcd-11d3-458c-97ec-1f7f8d399842 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:31.453890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffecdcd-11d3-458c-97ec-1f7f8d399842 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:31.460267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 491/1001] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Jul 20 15:13:31 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:13:32.205472 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fe254-6872-41a6-ae1d-b23c698cccd7 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:32.205843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fe254-6872-41a6-ae1d-b23c698cccd7 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:32.206042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fe254-6872-41a6-ae1d-b23c698cccd7 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:32.207581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 511/1002] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:32.212587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98376cc5-dd37-4ed2-8a64-584d0212aff8 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:32.213013 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98376cc5-dd37-4ed2-8a64-584d0212aff8 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:32.213174 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98376cc5-dd37-4ed2-8a64-584d0212aff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:32.230378 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-98376cc5-dd37-4ed2-8a64-584d0212aff8 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:32.231902 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-98376cc5-dd37-4ed2-8a64-584d0212aff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:32.300836 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 492/1003] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:32.308261 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977cfc3-0cca-4ae2-951c-07adb38734cd None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:32.308713 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977cfc3-0cca-4ae2-951c-07adb38734cd None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:32.308818 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977cfc3-0cca-4ae2-951c-07adb38734cd None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:32.310403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 512/1004] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:32.315648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8746c91-316a-4507-9a08-a437e04dfb85 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:32.315936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8746c91-316a-4507-9a08-a437e04dfb85 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:32.316213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8746c91-316a-4507-9a08-a437e04dfb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:32.324067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977cfc3-0cca-4ae2-951c-07adb38734cd None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:32.334825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-a8746c91-316a-4507-9a08-a437e04dfb85 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:32.336043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a8746c91-316a-4507-9a08-a437e04dfb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:32.387614 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:32.414001 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 493/1005] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:32.432300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977cfc3-0cca-4ae2-951c-07adb38734cd None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:32.436488 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8746c91-316a-4507-9a08-a437e04dfb85 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:32.445550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:32.470962 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4104ce-5d4e-49fe-a69d-bd032ec8cdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:32.472233 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4104ce-5d4e-49fe-a69d-bd032ec8cdf9 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:32.472852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4104ce-5d4e-49fe-a69d-bd032ec8cdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:32.473572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4104ce-5d4e-49fe-a69d-bd032ec8cdf9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:32.474238 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4104ce-5d4e-49fe-a69d-bd032ec8cdf9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:32.475654 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4104ce-5d4e-49fe-a69d-bd032ec8cdf9 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:32.488346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-ec4104ce-5d4e-49fe-a69d-bd032ec8cdf9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:13:32.488346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:32.488346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:32.488346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:32.488346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:32.488346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:32.488346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:32.488346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:32.488346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:13:32.488346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:32.488346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:32.488346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:32.488346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:32.488346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:32.488346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:32.488346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:32.488346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:32.489653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:32.489653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:32.489653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:32.489653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:32.489653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:32.489653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:32.489653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:32.489653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:32.489653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:32.489653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:32.489653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:32.489653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:32.489653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:32.489653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:32.489653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:32.489653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:32.490579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:32.490579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:32.490579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:32.490579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:32.490579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:32.490579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:32.490579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:32.490579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:32.490579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:32.490579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:32.490579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:32.490579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:32.490579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 15:13:32.490579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:32.490579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 513/1006] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Jul 20 15:13:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:32.495456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4104ce-5d4e-49fe-a69d-bd032ec8cdf9 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:32.552159 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8746c91-316a-4507-9a08-a437e04dfb85 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:32.589312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5931b04d-70cd-42e4-afb1-1163afbd782b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:32.589979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931b04d-70cd-42e4-afb1-1163afbd782b None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:32.590285 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931b04d-70cd-42e4-afb1-1163afbd782b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:32.590563 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931b04d-70cd-42e4-afb1-1163afbd782b None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:32.591290 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5931b04d-70cd-42e4-afb1-1163afbd782b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:32.592653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5931b04d-70cd-42e4-afb1-1163afbd782b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:32.606239 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4104ce-5d4e-49fe-a69d-bd032ec8cdf9 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:32.641753 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe3b29-c1ca-4164-8bee-f87d80bd9b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:32.642562 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe3b29-c1ca-4164-8bee-f87d80bd9b29 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:32.642844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe3b29-c1ca-4164-8bee-f87d80bd9b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:32.643120 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe3b29-c1ca-4164-8bee-f87d80bd9b29 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:32.643818 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe3b29-c1ca-4164-8bee-f87d80bd9b29 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:32.644128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe3b29-c1ca-4164-8bee-f87d80bd9b29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:32.645490 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe3b29-c1ca-4164-8bee-f87d80bd9b29 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:32.650295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 514/1007] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Jul 20 15:13:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:32.655799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe3b29-c1ca-4164-8bee-f87d80bd9b29 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:32.679956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 494/1008] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Jul 20 15:13:32 2022] POST /identity/v3/services => generated 242 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:13:32.705207 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:32.771316 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe3b29-c1ca-4164-8bee-f87d80bd9b29 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:32.813642 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38368ae6-1f22-4f94-9a1c-c2b2dcd62385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:32.814382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38368ae6-1f22-4f94-9a1c-c2b2dcd62385 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:32.814680 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38368ae6-1f22-4f94-9a1c-c2b2dcd62385 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:32.814949 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38368ae6-1f22-4f94-9a1c-c2b2dcd62385 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:32.815725 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38368ae6-1f22-4f94-9a1c-c2b2dcd62385 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:32.816038 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38368ae6-1f22-4f94-9a1c-c2b2dcd62385 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:32.817446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38368ae6-1f22-4f94-9a1c-c2b2dcd62385 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:32.828099 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 515/1009] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Jul 20 15:13:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:32.829214 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5931b04d-70cd-42e4-afb1-1163afbd782b None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:32.875986 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:32.936198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5931b04d-70cd-42e4-afb1-1163afbd782b None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:32.970831 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e15a2-850b-4dce-9014-1bc76478522f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:32.971494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e15a2-850b-4dce-9014-1bc76478522f None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:32.971778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e15a2-850b-4dce-9014-1bc76478522f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:32.972055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e15a2-850b-4dce-9014-1bc76478522f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:32.972795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942e15a2-850b-4dce-9014-1bc76478522f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:32.974167 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942e15a2-850b-4dce-9014-1bc76478522f None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:33.050111 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 495/1010] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Jul 20 15:13:32 2022] POST /identity/v3/endpoints => generated 325 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:13:33.493749 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716aa3b-9600-429e-b849-0017f2ea4783 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:33.494295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716aa3b-9600-429e-b849-0017f2ea4783 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:33.494414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716aa3b-9600-429e-b849-0017f2ea4783 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:33.495830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 516/1011] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:33.500736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fa979-6826-4db4-a503-eea91bbc8cc4 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:33.501111 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fa979-6826-4db4-a503-eea91bbc8cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:33.501600 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fa979-6826-4db4-a503-eea91bbc8cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:33.520124 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-e60fa979-6826-4db4-a503-eea91bbc8cc4 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:33.521325 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e60fa979-6826-4db4-a503-eea91bbc8cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:33.598406 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 496/1012] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:33 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:33.617758 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716aa3b-9600-429e-b849-0017f2ea4783 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:33.717535 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:33.722800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716aa3b-9600-429e-b849-0017f2ea4783 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:33.756896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b551752-3fb7-4fc2-b1e9-23ac7d2afe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:33.757629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b551752-3fb7-4fc2-b1e9-23ac7d2afe1a None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:33.757910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b551752-3fb7-4fc2-b1e9-23ac7d2afe1a None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:33.758186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b551752-3fb7-4fc2-b1e9-23ac7d2afe1a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:33.758934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b551752-3fb7-4fc2-b1e9-23ac7d2afe1a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:33.760371 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b551752-3fb7-4fc2-b1e9-23ac7d2afe1a None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:33.773079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-6b551752-3fb7-4fc2-b1e9-23ac7d2afe1a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:13:33.773079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:33.773079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:33.773079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:33.773079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:33.773079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:33.773079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:33.773079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:33.773079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:13:33.773079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:33.773079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:33.773079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:33.773079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:33.773079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:33.773079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:33.773079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:33.773079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:33.774143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:33.774143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:33.774143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:33.774143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:33.774143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:33.774143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:33.774143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:33.774143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:33.774143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:33.774143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:33.774143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:33.774143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:33.774143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:33.774143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:33.774143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:33.774143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:33.775076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:33.775076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:33.775076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:33.775076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:33.775076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:33.775076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:33.775076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:33.775076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:33.775076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:33.775076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:33.775076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:33.775076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:33.775076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 15:13:33.775076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:33.778609 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 517/1013] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:33.780492 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fa979-6826-4db4-a503-eea91bbc8cc4 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:33.808034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:33.889785 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fa979-6826-4db4-a503-eea91bbc8cc4 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:33.923965 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19f793-8b97-498c-ae08-59e4f5151447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:33.924874 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19f793-8b97-498c-ae08-59e4f5151447 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:33.925371 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19f793-8b97-498c-ae08-59e4f5151447 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:33.925830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19f793-8b97-498c-ae08-59e4f5151447 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:33.926735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc19f793-8b97-498c-ae08-59e4f5151447 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:33.927239 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc19f793-8b97-498c-ae08-59e4f5151447 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:33.928796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc19f793-8b97-498c-ae08-59e4f5151447 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:33.938799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 497/1014] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Jul 20 15:13:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:33.940492 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b551752-3fb7-4fc2-b1e9-23ac7d2afe1a None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:34.040129 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b551752-3fb7-4fc2-b1e9-23ac7d2afe1a None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:34.073302 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279d481-6313-4fe9-9513-5baef87e376c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:34.073965 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279d481-6313-4fe9-9513-5baef87e376c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:34.074250 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279d481-6313-4fe9-9513-5baef87e376c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:34.074553 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279d481-6313-4fe9-9513-5baef87e376c None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:34.075253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279d481-6313-4fe9-9513-5baef87e376c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:34.075561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279d481-6313-4fe9-9513-5baef87e376c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:34.077029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279d481-6313-4fe9-9513-5baef87e376c None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:34.087832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 518/1015] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Jul 20 15:13:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:34.087990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19f793-8b97-498c-ae08-59e4f5151447 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:34.182831 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:34.194067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19f793-8b97-498c-ae08-59e4f5151447 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:34.229424 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba97db-b74e-4082-830d-22ec5b5f6fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:34.230076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba97db-b74e-4082-830d-22ec5b5f6fba None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:34.230402 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba97db-b74e-4082-830d-22ec5b5f6fba None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:34.230683 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba97db-b74e-4082-830d-22ec5b5f6fba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:34.231442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba97db-b74e-4082-830d-22ec5b5f6fba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:34.232831 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba97db-b74e-4082-830d-22ec5b5f6fba None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:34.357504 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 498/1016] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Jul 20 15:13:34 2022] POST /identity/v3/endpoints => generated 334 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:13:34.749449 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590cb45-5a2e-4f41-b409-2162dabd6276 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:34.749935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590cb45-5a2e-4f41-b409-2162dabd6276 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:34.749935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590cb45-5a2e-4f41-b409-2162dabd6276 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:34.751955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 519/1017] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:34.757098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a2437-5310-415a-9bf9-4f5700480c41 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:34.757524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a2437-5310-415a-9bf9-4f5700480c41 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:34.757712 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a2437-5310-415a-9bf9-4f5700480c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:34.774870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-962a2437-5310-415a-9bf9-4f5700480c41 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:34.776057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-962a2437-5310-415a-9bf9-4f5700480c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:34.851985 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 499/1018] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:34.874895 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590cb45-5a2e-4f41-b409-2162dabd6276 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:34.905319 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:34.983301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590cb45-5a2e-4f41-b409-2162dabd6276 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:35.019980 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d8985-e613-4b79-ac50-4d136940ac40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:35.020778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d8985-e613-4b79-ac50-4d136940ac40 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:35.021063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d8985-e613-4b79-ac50-4d136940ac40 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:35.021342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d8985-e613-4b79-ac50-4d136940ac40 None admin] PATH_INFO: `/v3/services/image` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:35.022002 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d8985-e613-4b79-ac50-4d136940ac40 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:35.023493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d8985-e613-4b79-ac50-4d136940ac40 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:35.035887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-297d8985-e613-4b79-ac50-4d136940ac40 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:13:35.035887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:35.035887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:35.035887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:35.035887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:35.035887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:35.035887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:35.035887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:35.035887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:13:35.035887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:35.035887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:35.035887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:35.035887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:35.035887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:35.035887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:35.035887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:35.035887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:35.036913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:35.036913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:35.036913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:35.036913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:35.036913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:35.036913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:35.036913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:35.036913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:35.036913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:35.036913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:35.036913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:35.036913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:35.036913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:35.036913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:35.036913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:35.036913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:35.043212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:35.043212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:35.043212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:35.043212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:35.043212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:35.043212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:35.043212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:35.043212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:35.043212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:35.043212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:35.043212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:35.043212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:35.043212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:13:35.043212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:35.043212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 520/1019] 10.222.0.78 () {56 vars in 1105 bytes} [Wed Jul 20 15:13:34 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:35.044037 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a2437-5310-415a-9bf9-4f5700480c41 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:35.120746 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b407278-98ce-4079-8698-1ee7c605a50d None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:35.121047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b407278-98ce-4079-8698-1ee7c605a50d None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:35.121321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b407278-98ce-4079-8698-1ee7c605a50d None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:35.122541 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:35.122936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 521/1020] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:13:35.128742 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0d6b8-fb77-4644-b03b-509ce5fa4dfb None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:35.128980 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0d6b8-fb77-4644-b03b-509ce5fa4dfb None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:35.129260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0d6b8-fb77-4644-b03b-509ce5fa4dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:35.146089 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-3ce0d6b8-fb77-4644-b03b-509ce5fa4dfb None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:35.147317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-3ce0d6b8-fb77-4644-b03b-509ce5fa4dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:35.153122 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a2437-5310-415a-9bf9-4f5700480c41 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:35.188135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a1046-ae5e-41d6-8265-16522e6439c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:35.188802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a1046-ae5e-41d6-8265-16522e6439c9 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:35.189607 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a1046-ae5e-41d6-8265-16522e6439c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:35.189884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a1046-ae5e-41d6-8265-16522e6439c9 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:35.190675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a1046-ae5e-41d6-8265-16522e6439c9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:35.190986 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a1046-ae5e-41d6-8265-16522e6439c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:35.192352 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a1046-ae5e-41d6-8265-16522e6439c9 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:35.197926 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 500/1021] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:35.203365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a1046-ae5e-41d6-8265-16522e6439c9 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:35.222325 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 522/1022] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:13:35.247405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0d6b8-fb77-4644-b03b-509ce5fa4dfb None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:35.272164 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:35.309761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a1046-ae5e-41d6-8265-16522e6439c9 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:35.348721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af04f45-17d8-4703-b940-0d329fb00f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:35.349381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af04f45-17d8-4703-b940-0d329fb00f16 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:35.349662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af04f45-17d8-4703-b940-0d329fb00f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:35.349935 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af04f45-17d8-4703-b940-0d329fb00f16 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:35.350704 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af04f45-17d8-4703-b940-0d329fb00f16 None admin] RBAC: Adding query filter params (type=image) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:35.351010 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af04f45-17d8-4703-b940-0d329fb00f16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:35.351109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0d6b8-fb77-4644-b03b-509ce5fa4dfb None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:35.352367 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af04f45-17d8-4703-b940-0d329fb00f16 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:35.354485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:35.363597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 501/1023] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:35.364490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af04f45-17d8-4703-b940-0d329fb00f16 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:35.388585 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303011fa-d80c-4622-b2d4-0244b329f553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:35.389501 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303011fa-d80c-4622-b2d4-0244b329f553 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:35.389791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303011fa-d80c-4622-b2d4-0244b329f553 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:35.390096 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303011fa-d80c-4622-b2d4-0244b329f553 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:35.391305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303011fa-d80c-4622-b2d4-0244b329f553 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:35.392853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303011fa-d80c-4622-b2d4-0244b329f553 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:35.394514 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 523/1024] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:35.400108 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303011fa-d80c-4622-b2d4-0244b329f553 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:35.471590 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af04f45-17d8-4703-b940-0d329fb00f16 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:35.484924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:35.497732 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303011fa-d80c-4622-b2d4-0244b329f553 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:35.507524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bd363-f7c6-4d3f-b508-25db04622fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:35.508182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bd363-f7c6-4d3f-b508-25db04622fce None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:35.508465 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bd363-f7c6-4d3f-b508-25db04622fce None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:35.508749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bd363-f7c6-4d3f-b508-25db04622fce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:35.509534 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184bd363-f7c6-4d3f-b508-25db04622fce None admin] RBAC: Adding query filter params (interface=public, service_id=dba75601a67947baaaa7a8af51dcfb99, region_id=RegionOne) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:35.509851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184bd363-f7c6-4d3f-b508-25db04622fce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:35.511240 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184bd363-f7c6-4d3f-b508-25db04622fce None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:35.516458 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 502/1025] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Jul 20 15:13:35 2022] GET /identity/v3/endpoints?service_id=dba75601a67947baaaa7a8af51dcfb99&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:35.522784 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bd363-f7c6-4d3f-b508-25db04622fce None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:35.535793 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7242dc7-e268-4142-831c-9255eabe9720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:35.536464 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7242dc7-e268-4142-831c-9255eabe9720 None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:35.536746 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7242dc7-e268-4142-831c-9255eabe9720 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:35.537022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7242dc7-e268-4142-831c-9255eabe9720 None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:35.537732 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7242dc7-e268-4142-831c-9255eabe9720 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:35.539268 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7242dc7-e268-4142-831c-9255eabe9720 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:35.587201 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:35.624120 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bd363-f7c6-4d3f-b508-25db04622fce None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:35.628783 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 524/1026] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Jul 20 15:13:35 2022] POST /identity/v3/projects => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:13:35.662430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a6520-7906-4f74-b880-55a35c732430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:35.663116 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a6520-7906-4f74-b880-55a35c732430 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:35.663396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a6520-7906-4f74-b880-55a35c732430 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:35.663667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a6520-7906-4f74-b880-55a35c732430 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:35.664361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a6520-7906-4f74-b880-55a35c732430 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:35.665760 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a6520-7906-4f74-b880-55a35c732430 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:35.673898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 503/1027] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Jul 20 15:13:35 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:13:36.513427 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704e64e-6ac2-4765-b78c-1b3be870e09c None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:36.513922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704e64e-6ac2-4765-b78c-1b3be870e09c None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:36.514001 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704e64e-6ac2-4765-b78c-1b3be870e09c None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:36.515407 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 525/1028] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:36.520454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbebf2-615d-4b5b-8597-e6153d26db25 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:36.520895 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbebf2-615d-4b5b-8597-e6153d26db25 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:36.521043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbebf2-615d-4b5b-8597-e6153d26db25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:36.537871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-16cbebf2-615d-4b5b-8597-e6153d26db25 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:36.539093 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-16cbebf2-615d-4b5b-8597-e6153d26db25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:36.604022 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 504/1029] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:36.627806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704e64e-6ac2-4765-b78c-1b3be870e09c None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:36.728410 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704e64e-6ac2-4765-b78c-1b3be870e09c None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:36.762475 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b5542-8fa3-4637-8073-5f710bcb41a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:36.763294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b5542-8fa3-4637-8073-5f710bcb41a0 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:36.763625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b5542-8fa3-4637-8073-5f710bcb41a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:36.763852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b5542-8fa3-4637-8073-5f710bcb41a0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:36.771610 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:36.774981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b5542-8fa3-4637-8073-5f710bcb41a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:13:36.783934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-ac1b5542-8fa3-4637-8073-5f710bcb41a0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:13:36.783934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:36.783934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:36.783934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:36.783934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:36.783934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:36.783934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:36.783934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:36.783934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:36.783934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:36.783934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:36.783934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:36.783934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:13:36.783934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:13:36.783934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:13:36.783934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:13:36.784982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:13:36.784982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:13:36.784982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:13:36.784982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:13:36.784982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:36.784982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:36.784982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:36.784982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:36.784982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:36.784982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:36.784982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:36.784982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:36.784982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:36.784982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:36.784982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:36.784982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:36.785864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:36.785864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:36.785864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:36.785864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:36.785864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:13:36.785864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:13:36.785864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:13:36.785864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:13:36.785864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:13:36.785864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:13:36.785864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:13:36.785864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:36.793930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 526/1030] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Jul 20 15:13:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:36.794186 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbebf2-615d-4b5b-8597-e6153d26db25 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:36.854393 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:36.896974 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbebf2-615d-4b5b-8597-e6153d26db25 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:36.930234 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fccd37-67ec-4c39-a944-46d8a63994f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:36.930928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fccd37-67ec-4c39-a944-46d8a63994f2 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:36.931216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fccd37-67ec-4c39-a944-46d8a63994f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:36.931496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fccd37-67ec-4c39-a944-46d8a63994f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:36.932246 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fccd37-67ec-4c39-a944-46d8a63994f2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:36.932556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fccd37-67ec-4c39-a944-46d8a63994f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:36.933924 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fccd37-67ec-4c39-a944-46d8a63994f2 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:36.942062 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 505/1031] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Jul 20 15:13:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:36.947551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b5542-8fa3-4637-8073-5f710bcb41a0 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:37.047478 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b5542-8fa3-4637-8073-5f710bcb41a0 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:37.081901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611edac-52c8-49b5-87ee-b120c8c0fbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:37.082739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611edac-52c8-49b5-87ee-b120c8c0fbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:37.083026 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611edac-52c8-49b5-87ee-b120c8c0fbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:37.083305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611edac-52c8-49b5-87ee-b120c8c0fbd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:37.099841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611edac-52c8-49b5-87ee-b120c8c0fbd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:37.101298 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611edac-52c8-49b5-87ee-b120c8c0fbd5 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:37.118867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-6611edac-52c8-49b5-87ee-b120c8c0fbd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:37.118867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:37.118867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:37.118867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:37.118867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:37.118867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:37.118867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:37.118867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:37.118867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:37.118867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:37.118867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:37.118867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:37.118867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:37.118867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:37.118867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:37.119812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:37.119812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:37.119812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:37.119812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:37.119812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:37.119812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:37.119812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:37.119812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:37.119812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:37.119812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:37.119812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:37.119812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:37.119812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:37.119812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:37.119812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:37.119812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:37.119812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:37.120786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:37.120786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:37.120786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:37.120786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:37.121581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 527/1032] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:37.125764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fccd37-67ec-4c39-a944-46d8a63994f2 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:37.228535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fccd37-67ec-4c39-a944-46d8a63994f2 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:37.248536 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:37.263614 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cf81e-c9f9-4189-9751-d92001d0bafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:37.264285 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cf81e-c9f9-4189-9751-d92001d0bafe None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:37.264574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cf81e-c9f9-4189-9751-d92001d0bafe None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:37.264862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cf81e-c9f9-4189-9751-d92001d0bafe None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:37.269271 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6cf81e-c9f9-4189-9751-d92001d0bafe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:37.269271 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6cf81e-c9f9-4189-9751-d92001d0bafe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:37.269271 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6cf81e-c9f9-4189-9751-d92001d0bafe None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:37.284224 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 506/1033] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:37.285538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611edac-52c8-49b5-87ee-b120c8c0fbd5 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:37.342024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:37.391370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611edac-52c8-49b5-87ee-b120c8c0fbd5 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:37.425666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6017bd0-ede0-4d06-adb9-194d8e8a2d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:37.426506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6017bd0-ede0-4d06-adb9-194d8e8a2d0e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:37.426782 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6017bd0-ede0-4d06-adb9-194d8e8a2d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:37.427055 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6017bd0-ede0-4d06-adb9-194d8e8a2d0e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:37.434700 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6017bd0-ede0-4d06-adb9-194d8e8a2d0e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:37.436148 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6017bd0-ede0-4d06-adb9-194d8e8a2d0e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:37.449921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-f6017bd0-ede0-4d06-adb9-194d8e8a2d0e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:13:37.449921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:37.449921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:37.449921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:37.449921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:37.449921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:37.449921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:37.449921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:37.449921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:37.449921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:37.449921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:37.449921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:37.449921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:13:37.449921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:13:37.449921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:13:37.449921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:37.451116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:37.452123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:37.452123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:37.452123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:13:37.452123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:13:37.452123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:13:37.452123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:13:37.452123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:13:37.452123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:13:37.452123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:13:37.452123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:37.453240 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 528/1034] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Jul 20 15:13:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:37.456834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cf81e-c9f9-4189-9751-d92001d0bafe None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:37.560439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cf81e-c9f9-4189-9751-d92001d0bafe None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:37.595350 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8e4b6-f80c-4702-9524-9c3eb4031e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:37.595954 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8e4b6-f80c-4702-9524-9c3eb4031e86 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:37.596239 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8e4b6-f80c-4702-9524-9c3eb4031e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:37.596517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8e4b6-f80c-4702-9524-9c3eb4031e86 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:37.597354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8e4b6-f80c-4702-9524-9c3eb4031e86 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:37.597668 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8e4b6-f80c-4702-9524-9c3eb4031e86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:37.599126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8e4b6-f80c-4702-9524-9c3eb4031e86 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:37.616377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 507/1035] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Jul 20 15:13:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:37.621872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6017bd0-ede0-4d06-adb9-194d8e8a2d0e None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:37.721675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c47e75-a72a-46ff-8467-bd7c17d20a42 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:37.722331 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c47e75-a72a-46ff-8467-bd7c17d20a42 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:37.722672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c47e75-a72a-46ff-8467-bd7c17d20a42 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:37.724063 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 508/1036] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:13:37.728573 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6017bd0-ede0-4d06-adb9-194d8e8a2d0e None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:37.729104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39cb88-2529-4f81-b913-f0e102e6c662 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:37.729397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39cb88-2529-4f81-b913-f0e102e6c662 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:37.729730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39cb88-2529-4f81-b913-f0e102e6c662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:37.742445 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:37.746714 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-1f39cb88-2529-4f81-b913-f0e102e6c662 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:37.747910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-1f39cb88-2529-4f81-b913-f0e102e6c662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:37.763846 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cdf14-6ed0-4643-9c9a-b8e6a344473d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:37.764497 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cdf14-6ed0-4643-9c9a-b8e6a344473d None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:37.764783 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cdf14-6ed0-4643-9c9a-b8e6a344473d None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:37.765067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cdf14-6ed0-4643-9c9a-b8e6a344473d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:37.777928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:37.815496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 509/1037] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:37.816720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff0acb-e27d-4bd6-b1e3-236754a7d4a7 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:37.816992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff0acb-e27d-4bd6-b1e3-236754a7d4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:37.817316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff0acb-e27d-4bd6-b1e3-236754a7d4a7 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:37.822699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 510/1038] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:37.823822 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3c238-e755-4b92-95eb-64cd0f9707c2 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:37.824114 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3c238-e755-4b92-95eb-64cd0f9707c2 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:37.824396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3c238-e755-4b92-95eb-64cd0f9707c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:37.835127 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790cdf14-6ed0-4643-9c9a-b8e6a344473d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:37.836631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790cdf14-6ed0-4643-9c9a-b8e6a344473d None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:37.840773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-1bf3c238-e755-4b92-95eb-64cd0f9707c2 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:37.841952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-1bf3c238-e755-4b92-95eb-64cd0f9707c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:37.908748 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 511/1039] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:37.909500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3c238-e755-4b92-95eb-64cd0f9707c2 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:37.935063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 529/1040] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:13:37 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 15:13:37.935928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cdf14-6ed0-4643-9c9a-b8e6a344473d None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:37.949965 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:38.012267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:38.016263 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3c238-e755-4b92-95eb-64cd0f9707c2 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:38.031545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cdf14-6ed0-4643-9c9a-b8e6a344473d None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:38.049214 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe763bfb-d705-4471-ba66-0e859b9df76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:38.049993 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe763bfb-d705-4471-ba66-0e859b9df76f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:38.050300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe763bfb-d705-4471-ba66-0e859b9df76f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:38.050568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe763bfb-d705-4471-ba66-0e859b9df76f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:38.051727 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe763bfb-d705-4471-ba66-0e859b9df76f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:38.053135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe763bfb-d705-4471-ba66-0e859b9df76f None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:38.054784 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 512/1041] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:38.055530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe763bfb-d705-4471-ba66-0e859b9df76f None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:38.064495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afe3a5-83a9-4431-86d4-ca7d28fe7d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:38.065283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afe3a5-83a9-4431-86d4-ca7d28fe7d2e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:38.065566 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afe3a5-83a9-4431-86d4-ca7d28fe7d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:38.065840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afe3a5-83a9-4431-86d4-ca7d28fe7d2e None admin] PATH_INFO: `/v3/services/image` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:38.066521 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afe3a5-83a9-4431-86d4-ca7d28fe7d2e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:38.067889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afe3a5-83a9-4431-86d4-ca7d28fe7d2e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:38.079186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-57afe3a5-83a9-4431-86d4-ca7d28fe7d2e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:13:38.079186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:38.079186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:13:38.079186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:13:38.079186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:13:38.079186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:13:38.079186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:13:38.079186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:38.079186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:13:38.079186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:38.079186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:13:38.079186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:38.079186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:38.079186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:38.079186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:38.079186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:38.079186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:38.080805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:38.080805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:38.080805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:38.080805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:38.080805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:38.080805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:38.080805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:13:38.080805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:13:38.080805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:13:38.080805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:13:38.080805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:38.080805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:38.080805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:38.080805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:38.080805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:38.080805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:38.081739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:38.081739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:38.081739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:38.081739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:38.081739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:38.081739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:38.081739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:38.081739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:38.081739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:38.081739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:38.081739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:13:38.081739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:13:38.081739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 15:13:38.081739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:38.081739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 530/1042] 10.222.0.78 () {56 vars in 1105 bytes} [Wed Jul 20 15:13:37 2022] GET /identity/v3/services/image => generated 86 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:38.082620 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afe3a5-83a9-4431-86d4-ca7d28fe7d2e None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:38.164451 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe763bfb-d705-4471-ba66-0e859b9df76f None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:38.165867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:38.187125 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afe3a5-83a9-4431-86d4-ca7d28fe7d2e None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:38.198882 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19212d9-7413-4900-858e-74645caaf3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:38.199673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19212d9-7413-4900-858e-74645caaf3de None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:38.199959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19212d9-7413-4900-858e-74645caaf3de None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:38.200243 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19212d9-7413-4900-858e-74645caaf3de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:38.210270 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19212d9-7413-4900-858e-74645caaf3de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:38.211720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19212d9-7413-4900-858e-74645caaf3de None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:38.220013 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9d85e-8208-492c-b5da-643087497f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:38.221176 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9d85e-8208-492c-b5da-643087497f85 None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:38.221462 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9d85e-8208-492c-b5da-643087497f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:38.221743 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9d85e-8208-492c-b5da-643087497f85 None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:38.222489 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9d85e-8208-492c-b5da-643087497f85 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:38.223908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9d85e-8208-492c-b5da-643087497f85 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:38.229274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-a19212d9-7413-4900-858e-74645caaf3de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:38.229274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:38.229274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:38.229274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:38.229274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:38.229274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:38.229274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:38.229274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:38.229274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:38.229274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:38.229274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:38.229274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:38.229274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:38.229274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:38.229274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:38.230361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:38.230361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:38.230361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:38.230361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:38.230361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:38.230361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:38.230361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:38.230361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:38.230361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:38.230361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:38.230361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:38.230361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:38.230361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:38.230361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:38.230361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:38.230361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:38.230361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:38.231304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:38.231304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:38.231304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:38.231304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:38.231304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 513/1043] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:38.231304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19212d9-7413-4900-858e-74645caaf3de None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:38.250355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:38.305314 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 531/1044] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:13:38 2022] POST /identity/v3/users => generated 285 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:13:38.306154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9d85e-8208-492c-b5da-643087497f85 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:38.333669 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19212d9-7413-4900-858e-74645caaf3de None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:38.380669 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e71add-8cf5-4996-b640-98e97bcfb8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:38.382191 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e71add-8cf5-4996-b640-98e97bcfb8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:38.382872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e71add-8cf5-4996-b640-98e97bcfb8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:38.383408 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e71add-8cf5-4996-b640-98e97bcfb8d1 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:38.384469 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e71add-8cf5-4996-b640-98e97bcfb8d1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:38.384945 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e71add-8cf5-4996-b640-98e97bcfb8d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:38.387183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e71add-8cf5-4996-b640-98e97bcfb8d1 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:38.395944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 514/1045] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:38.402475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e71add-8cf5-4996-b640-98e97bcfb8d1 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:38.434096 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9d85e-8208-492c-b5da-643087497f85 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:38.471413 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:38.481134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10799c-0601-46ac-be35-3ba91b4d617f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:38.482146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10799c-0601-46ac-be35-3ba91b4d617f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:38.482146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10799c-0601-46ac-be35-3ba91b4d617f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:38.482410 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10799c-0601-46ac-be35-3ba91b4d617f None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:38.483113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10799c-0601-46ac-be35-3ba91b4d617f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:38.483420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10799c-0601-46ac-be35-3ba91b4d617f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:38.484811 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10799c-0601-46ac-be35-3ba91b4d617f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:38.498069 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 532/1046] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:38.502641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:38.504844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10799c-0601-46ac-be35-3ba91b4d617f None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:38.522971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e71add-8cf5-4996-b640-98e97bcfb8d1 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:38.565328 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705246b-7df3-4d66-95e2-bd7a43f00985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:38.566076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705246b-7df3-4d66-95e2-bd7a43f00985 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:38.566303 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705246b-7df3-4d66-95e2-bd7a43f00985 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:38.566567 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705246b-7df3-4d66-95e2-bd7a43f00985 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:38.567399 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705246b-7df3-4d66-95e2-bd7a43f00985 None admin] RBAC: Adding query filter params (type=image) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:38.567705 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705246b-7df3-4d66-95e2-bd7a43f00985 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:38.569111 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705246b-7df3-4d66-95e2-bd7a43f00985 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:38.574073 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 515/1047] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:38.579425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705246b-7df3-4d66-95e2-bd7a43f00985 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:38.615260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10799c-0601-46ac-be35-3ba91b4d617f None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:38.651740 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f9479-1727-4c2f-a980-8bdfa158e646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:38.652538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f9479-1727-4c2f-a980-8bdfa158e646 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:38.652814 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f9479-1727-4c2f-a980-8bdfa158e646 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:38.653087 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f9479-1727-4c2f-a980-8bdfa158e646 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:38.660051 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f9479-1727-4c2f-a980-8bdfa158e646 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:38.661484 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f9479-1727-4c2f-a980-8bdfa158e646 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:38.676021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-7f3f9479-1727-4c2f-a980-8bdfa158e646 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:38.676021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:38.676021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:38.676021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:38.676021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:38.676021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:38.676021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:38.676021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:38.676021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:38.676021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:38.676021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:38.676021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:38.676021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:13:38.676021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:13:38.676021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:13:38.677185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:13:38.677185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:38.677185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:38.677185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:38.677185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:38.677185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:38.677185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:38.677185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:38.677185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:38.677185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:38.677185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:38.677185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:38.677185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:38.677185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:38.677185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:38.677185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:38.677185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:38.678328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:13:38.678328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:13:38.678328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:13:38.678328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:13:38.678328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:13:38.678328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:13:38.678328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:38.678328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:38.682071 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 533/1048] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:38.682840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f9479-1727-4c2f-a980-8bdfa158e646 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:38.689972 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705246b-7df3-4d66-95e2-bd7a43f00985 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:38.729154 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8558a-e04f-4983-a523-1ee5c83099e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:38.729966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8558a-e04f-4983-a523-1ee5c83099e6 None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:38.730730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8558a-e04f-4983-a523-1ee5c83099e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:38.731118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8558a-e04f-4983-a523-1ee5c83099e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:38.732295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8558a-e04f-4983-a523-1ee5c83099e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:38.734194 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8558a-e04f-4983-a523-1ee5c83099e6 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:38.741358 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:38.788331 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f9479-1727-4c2f-a980-8bdfa158e646 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:38.814101 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 516/1049] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Jul 20 15:13:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:13:38.832126 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aade84-7404-445a-b780-fa0912fb2a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:38.832815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aade84-7404-445a-b780-fa0912fb2a65 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:38.833094 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aade84-7404-445a-b780-fa0912fb2a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:38.833367 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aade84-7404-445a-b780-fa0912fb2a65 None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:38.834173 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aade84-7404-445a-b780-fa0912fb2a65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:38.834589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aade84-7404-445a-b780-fa0912fb2a65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:38.836068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aade84-7404-445a-b780-fa0912fb2a65 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:38.850975 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 534/1050] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:13:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:38.857186 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8558a-e04f-4983-a523-1ee5c83099e6 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:38.860572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:38.972300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8558a-e04f-4983-a523-1ee5c83099e6 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:39.007887 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6dac6-eedc-4099-b081-cf4166e2c9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:39.008547 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6dac6-eedc-4099-b081-cf4166e2c9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:39.008833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6dac6-eedc-4099-b081-cf4166e2c9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:39.009115 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6dac6-eedc-4099-b081-cf4166e2c9a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:39.009954 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6dac6-eedc-4099-b081-cf4166e2c9a9 None admin] RBAC: Adding query filter params (role.id=e2d60ba4ec674cbb8710971ec3f9bdfe, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=a4a24d5a375945779fa626fae25ac237) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:39.010303 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6dac6-eedc-4099-b081-cf4166e2c9a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:39.011843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6dac6-eedc-4099-b081-cf4166e2c9a9 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:39.017758 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 517/1051] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:13:38 2022] GET /identity/v3/role_assignments?user.id=a4a24d5a375945779fa626fae25ac237&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=e2d60ba4ec674cbb8710971ec3f9bdfe => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:40.407090 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cc8a5-135b-4d48-98e5-2fee3c4d7fc1 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:40.408091 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cc8a5-135b-4d48-98e5-2fee3c4d7fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:40.408091 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cc8a5-135b-4d48-98e5-2fee3c4d7fc1 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:40.409612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 535/1052] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:40.414536 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45286377-bcd2-4618-bccf-b367fc8f949f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:40.414914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45286377-bcd2-4618-bccf-b367fc8f949f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:40.415092 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45286377-bcd2-4618-bccf-b367fc8f949f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:40.432109 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-45286377-bcd2-4618-bccf-b367fc8f949f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:40.433279 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-45286377-bcd2-4618-bccf-b367fc8f949f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:40.494375 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:40.502813 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 518/1053] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:40.526222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6cc8a5-135b-4d48-98e5-2fee3c4d7fc1 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:40.626985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6cc8a5-135b-4d48-98e5-2fee3c4d7fc1 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:40.636497 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:40.661270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678e509-4002-46d0-b7ce-b5ef7ab70321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:40.661982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678e509-4002-46d0-b7ce-b5ef7ab70321 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:40.662289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678e509-4002-46d0-b7ce-b5ef7ab70321 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:40.662556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678e509-4002-46d0-b7ce-b5ef7ab70321 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:40.673100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678e509-4002-46d0-b7ce-b5ef7ab70321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:13:40.680735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-2678e509-4002-46d0-b7ce-b5ef7ab70321 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:13:40.680735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:40.680735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:40.680735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:40.680735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:40.680735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:40.680735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:40.680735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:40.680735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:40.680735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:40.680735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:40.680735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:40.680735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:13:40.680735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:13:40.680735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:13:40.680735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:13:40.681671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:13:40.681671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:13:40.681671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:13:40.681671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:13:40.681671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:40.681671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:40.681671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:40.681671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:40.681671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:40.681671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:40.681671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:40.681671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:40.681671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:40.681671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:40.681671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:40.681671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:40.683079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:40.683079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:40.683079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:40.683079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:40.683079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:13:40.683079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:13:40.683079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:13:40.683079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:13:40.683079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:13:40.683079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:13:40.683079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:13:40.683079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:40.686672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 536/1054] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:13:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:40.687882 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45286377-bcd2-4618-bccf-b367fc8f949f None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:40.794896 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45286377-bcd2-4618-bccf-b367fc8f949f None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:40.829867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9315b-82a6-43b0-b186-3f41566abe4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:40.830584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9315b-82a6-43b0-b186-3f41566abe4b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:40.830865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9315b-82a6-43b0-b186-3f41566abe4b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:40.831135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9315b-82a6-43b0-b186-3f41566abe4b None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:40.831871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9315b-82a6-43b0-b186-3f41566abe4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:40.832175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9315b-82a6-43b0-b186-3f41566abe4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:40.833534 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9315b-82a6-43b0-b186-3f41566abe4b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:40.846282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 519/1055] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:13:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:40.847876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678e509-4002-46d0-b7ce-b5ef7ab70321 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:40.873947 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c0777-7af8-4099-9b64-3737e9f6d2a2 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:40.874293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c0777-7af8-4099-9b64-3737e9f6d2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:40.874555 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c0777-7af8-4099-9b64-3737e9f6d2a2 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:40.876959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 520/1056] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:13:40.881618 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba66765-1f49-4788-8d57-9c89ed4e64e2 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:40.881915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba66765-1f49-4788-8d57-9c89ed4e64e2 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:40.882194 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba66765-1f49-4788-8d57-9c89ed4e64e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:40.901182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-cba66765-1f49-4788-8d57-9c89ed4e64e2 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:40.903029 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-cba66765-1f49-4788-8d57-9c89ed4e64e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:40.954957 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678e509-4002-46d0-b7ce-b5ef7ab70321 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:40.966560 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:40.980857 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 521/1057] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:13:40.991700 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06613002-ba24-4001-aac6-47a0f384e412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:40.992568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06613002-ba24-4001-aac6-47a0f384e412 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:40.992855 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06613002-ba24-4001-aac6-47a0f384e412 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:40.993123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06613002-ba24-4001-aac6-47a0f384e412 None admin] PATH_INFO: `/v3/users/0c6582746c484d378e0dce098f10456b` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:41.004245 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba66765-1f49-4788-8d57-9c89ed4e64e2 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:41.007894 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06613002-ba24-4001-aac6-47a0f384e412 None admin] RBAC: Authorizing `identity:get_user(user_id=0c6582746c484d378e0dce098f10456b)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:41.009335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06613002-ba24-4001-aac6-47a0f384e412 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:41.015882 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 537/1058] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:13:40 2022] GET /identity/v3/users/0c6582746c484d378e0dce098f10456b => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:41.016499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06613002-ba24-4001-aac6-47a0f384e412 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:41.043675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:41.111713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba66765-1f49-4788-8d57-9c89ed4e64e2 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:41.116736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06613002-ba24-4001-aac6-47a0f384e412 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:41.145199 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39481d-feaa-42b1-993c-24efd9cabd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:41.145841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39481d-feaa-42b1-993c-24efd9cabd70 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:41.146127 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39481d-feaa-42b1-993c-24efd9cabd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:41.146428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39481d-feaa-42b1-993c-24efd9cabd70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:41.149955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639fc29-6b96-4649-8982-2de07582c06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:41.150652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639fc29-6b96-4649-8982-2de07582c06b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:41.150915 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639fc29-6b96-4649-8982-2de07582c06b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:41.151194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639fc29-6b96-4649-8982-2de07582c06b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:41.200343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:41.212598 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39481d-feaa-42b1-993c-24efd9cabd70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:41.214144 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39481d-feaa-42b1-993c-24efd9cabd70 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:41.216969 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639fc29-6b96-4649-8982-2de07582c06b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:41.218418 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639fc29-6b96-4649-8982-2de07582c06b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:41.243839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:41.310752 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 522/1059] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:13:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 15:13:41.312145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd4c9f-4ba0-4ea0-a37f-e51c1c0ca1de None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:41.312436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd4c9f-4ba0-4ea0-a37f-e51c1c0ca1de None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:41.312714 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd4c9f-4ba0-4ea0-a37f-e51c1c0ca1de None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:41.318592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 523/1060] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:41.318592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd4c9f-4ba0-4ea0-a37f-e51c1c0ca1de None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:41.318838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 538/1061] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:13:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 15:13:41.322650 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639fc29-6b96-4649-8982-2de07582c06b None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:41.412674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:41.424463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639fc29-6b96-4649-8982-2de07582c06b None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:41.425121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd4c9f-4ba0-4ea0-a37f-e51c1c0ca1de None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:41.449556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:41.458621 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbd051-7f4c-4c0e-93d9-e608f1f42d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:41.459523 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbd051-7f4c-4c0e-93d9-e608f1f42d96 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:41.460351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860a95c-4918-4787-a391-7328a46d7110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:41.460479 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbd051-7f4c-4c0e-93d9-e608f1f42d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:41.460479 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbd051-7f4c-4c0e-93d9-e608f1f42d96 None admin] PATH_INFO: `/v3/projects/af45393e1609453aa5bbbc460146d8e5` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:41.460657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860a95c-4918-4787-a391-7328a46d7110 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:41.460742 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860a95c-4918-4787-a391-7328a46d7110 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:41.460994 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860a95c-4918-4787-a391-7328a46d7110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:41.461543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbbd051-7f4c-4c0e-93d9-e608f1f42d96 None admin] RBAC: Authorizing `identity:get_project(project_id=af45393e1609453aa5bbbc460146d8e5)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:41.463060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbbd051-7f4c-4c0e-93d9-e608f1f42d96 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:41.464689 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 539/1062] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:13:41 2022] GET /identity/v3/projects/af45393e1609453aa5bbbc460146d8e5 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:41.465957 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc885210-0b9c-4cc2-b0cb-99218f6913ca None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:41.466240 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc885210-0b9c-4cc2-b0cb-99218f6913ca None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:41.466540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc885210-0b9c-4cc2-b0cb-99218f6913ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:41.473606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860a95c-4918-4787-a391-7328a46d7110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:41.475094 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860a95c-4918-4787-a391-7328a46d7110 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:41.482348 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-bc885210-0b9c-4cc2-b0cb-99218f6913ca None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:41.483911 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-bc885210-0b9c-4cc2-b0cb-99218f6913ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:41.493272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-5860a95c-4918-4787-a391-7328a46d7110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:41.493272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:41.493272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:41.493272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:41.493272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:41.493272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:41.493272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:41.493272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:41.493272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:41.493272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:41.493272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:41.493272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:41.493272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:41.493272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:41.493272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:41.500572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:41.500572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:41.500572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:41.500572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:41.500572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:41.500572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:41.500572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:41.500572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:41.500572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:41.500572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:41.500572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:41.500572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:41.500572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:41.500572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:41.500572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:41.500572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:41.500572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:41.502627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:41.502627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:41.502627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:41.502627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:41.502627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 524/1063] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:41.502627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860a95c-4918-4787-a391-7328a46d7110 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:41.558569 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 540/1064] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:41.559418 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc885210-0b9c-4cc2-b0cb-99218f6913ca None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:41.594465 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860a95c-4918-4787-a391-7328a46d7110 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:41.633228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3a6e1-928d-4f02-803d-018a4d3c6669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:41.633873 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3a6e1-928d-4f02-803d-018a4d3c6669 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:41.634157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3a6e1-928d-4f02-803d-018a4d3c6669 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:41.634458 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3a6e1-928d-4f02-803d-018a4d3c6669 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:41.635536 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3a6e1-928d-4f02-803d-018a4d3c6669 None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=af45393e1609453aa5bbbc460146d8e5, user.id=0c6582746c484d378e0dce098f10456b) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:41.635844 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3a6e1-928d-4f02-803d-018a4d3c6669 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:41.637403 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3a6e1-928d-4f02-803d-018a4d3c6669 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:41.644417 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 525/1065] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:13:41 2022] GET /identity/v3/role_assignments?user.id=0c6582746c484d378e0dce098f10456b&scope.project.id=af45393e1609453aa5bbbc460146d8e5&role.id=25516be1485c49538544ea8a596e7d17 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:41.645206 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3a6e1-928d-4f02-803d-018a4d3c6669 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:41.648778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:41.682574 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc885210-0b9c-4cc2-b0cb-99218f6913ca None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:41.711100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:41.722544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67db5e0-f7c6-4bf8-8828-baa7cfd7ae35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:41.723147 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67db5e0-f7c6-4bf8-8828-baa7cfd7ae35 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:41.723438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67db5e0-f7c6-4bf8-8828-baa7cfd7ae35 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:41.723712 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67db5e0-f7c6-4bf8-8828-baa7cfd7ae35 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:41.724453 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67db5e0-f7c6-4bf8-8828-baa7cfd7ae35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:41.724762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67db5e0-f7c6-4bf8-8828-baa7cfd7ae35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:41.726242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67db5e0-f7c6-4bf8-8828-baa7cfd7ae35 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:41.750294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 541/1066] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:41.761027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3a6e1-928d-4f02-803d-018a4d3c6669 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:41.802042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225a45b-0f66-4432-beef-d66f47aca42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:41.802910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225a45b-0f66-4432-beef-d66f47aca42d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:41.803182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225a45b-0f66-4432-beef-d66f47aca42d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:41.803458 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225a45b-0f66-4432-beef-d66f47aca42d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:41.815552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225a45b-0f66-4432-beef-d66f47aca42d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:13:41.824074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-8225a45b-0f66-4432-beef-d66f47aca42d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:13:41.824074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:41.824074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:41.824074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:41.824074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:41.824074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:41.824074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:41.824074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:41.824074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:41.824074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:41.824074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:41.824074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:41.824074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:13:41.824074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:13:41.824074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:13:41.824074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:13:41.825172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:13:41.825172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:13:41.825172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:13:41.825172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:13:41.825172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:41.825172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:41.825172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:41.825172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:41.825172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:41.825172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:41.825172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:41.825172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:41.825172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:41.825172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:41.825172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:41.825172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:41.826077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:41.826077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:41.826077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:41.826077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:41.826077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:13:41.826077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:13:41.826077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:13:41.826077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:13:41.826077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:13:41.826077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:13:41.826077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:13:41.826077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:41.826077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 526/1067] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Jul 20 15:13:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:41.834397 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67db5e0-f7c6-4bf8-8828-baa7cfd7ae35 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:41.942422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67db5e0-f7c6-4bf8-8828-baa7cfd7ae35 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:41.978964 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9661ad-79aa-4115-a7e4-d0b1e3daffe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:41.980391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9661ad-79aa-4115-a7e4-d0b1e3daffe4 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:41.980391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9661ad-79aa-4115-a7e4-d0b1e3daffe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:41.980391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9661ad-79aa-4115-a7e4-d0b1e3daffe4 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:41.980834 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9661ad-79aa-4115-a7e4-d0b1e3daffe4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:41.981144 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9661ad-79aa-4115-a7e4-d0b1e3daffe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:41.982531 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9661ad-79aa-4115-a7e4-d0b1e3daffe4 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:41.991176 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 542/1068] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Jul 20 15:13:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:41.997110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225a45b-0f66-4432-beef-d66f47aca42d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:42.066706 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:42.105772 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225a45b-0f66-4432-beef-d66f47aca42d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:42.139912 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfa626-e5d8-4cc3-b7e8-427e4c24b04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:42.140703 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfa626-e5d8-4cc3-b7e8-427e4c24b04b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:42.140987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfa626-e5d8-4cc3-b7e8-427e4c24b04b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:42.141263 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfa626-e5d8-4cc3-b7e8-427e4c24b04b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:42.152323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfa626-e5d8-4cc3-b7e8-427e4c24b04b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:42.153768 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfa626-e5d8-4cc3-b7e8-427e4c24b04b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:42.170526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-29bfa626-e5d8-4cc3-b7e8-427e4c24b04b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:42.170526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:42.170526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:42.170526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:42.170526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:42.170526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:42.170526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:42.170526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:42.170526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:42.170526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:42.170526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:42.170526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:42.170526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:42.170526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:42.170526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:42.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:42.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:42.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:42.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:42.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:42.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:42.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:42.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:42.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:42.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:42.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:42.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:42.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:42.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:42.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:42.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:42.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:42.172490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:42.172490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:42.172490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:42.172490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:42.172490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 527/1069] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:42.177539 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9661ad-79aa-4115-a7e4-d0b1e3daffe4 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:42.223604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:42.280058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9661ad-79aa-4115-a7e4-d0b1e3daffe4 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:42.313963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760241c4-9056-43b9-ba95-386028c256d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:42.314668 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760241c4-9056-43b9-ba95-386028c256d2 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:42.314954 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760241c4-9056-43b9-ba95-386028c256d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:42.315236 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760241c4-9056-43b9-ba95-386028c256d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:42.315956 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760241c4-9056-43b9-ba95-386028c256d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:42.316265 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760241c4-9056-43b9-ba95-386028c256d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:42.317638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760241c4-9056-43b9-ba95-386028c256d2 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:42.329936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 543/1070] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:42.335521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfa626-e5d8-4cc3-b7e8-427e4c24b04b None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:42.432256 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfa626-e5d8-4cc3-b7e8-427e4c24b04b None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:42.447452 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:42.464745 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657687b-064e-4573-981b-c21861ff21f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:42.465529 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657687b-064e-4573-981b-c21861ff21f0 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:42.465810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657687b-064e-4573-981b-c21861ff21f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:42.466084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657687b-064e-4573-981b-c21861ff21f0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:42.473293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657687b-064e-4573-981b-c21861ff21f0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:42.474851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657687b-064e-4573-981b-c21861ff21f0 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:42.489394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-5657687b-064e-4573-981b-c21861ff21f0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:13:42.489394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:42.489394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:42.489394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:42.489394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:42.489394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:42.489394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:42.489394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:42.489394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:42.489394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:42.489394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:42.489394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:42.489394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:13:42.489394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:13:42.489394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:13:42.489394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:42.490440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:42.491439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:42.491439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:42.491439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:13:42.491439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:13:42.491439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:13:42.491439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:13:42.491439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:13:42.491439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:13:42.491439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:13:42.491439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:42.494421 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 528/1071] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Jul 20 15:13:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:42.496280 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760241c4-9056-43b9-ba95-386028c256d2 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:42.593400 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760241c4-9056-43b9-ba95-386028c256d2 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:42.600596 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:42.626868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cdefac-d010-498e-8730-99171f803fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:42.627562 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdefac-d010-498e-8730-99171f803fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:42.627841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdefac-d010-498e-8730-99171f803fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:42.628109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdefac-d010-498e-8730-99171f803fc6 None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:42.628926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cdefac-d010-498e-8730-99171f803fc6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:42.629227 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cdefac-d010-498e-8730-99171f803fc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:42.630665 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cdefac-d010-498e-8730-99171f803fc6 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:42.647489 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 544/1072] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Jul 20 15:13:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:42.653084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657687b-064e-4573-981b-c21861ff21f0 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:42.750824 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657687b-064e-4573-981b-c21861ff21f0 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:42.785179 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6bd71-168e-4bbf-9559-61e45ffb6562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:42.785968 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6bd71-168e-4bbf-9559-61e45ffb6562 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:42.786249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6bd71-168e-4bbf-9559-61e45ffb6562 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:42.786545 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6bd71-168e-4bbf-9559-61e45ffb6562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:42.796953 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6bd71-168e-4bbf-9559-61e45ffb6562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:42.798983 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6bd71-168e-4bbf-9559-61e45ffb6562 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:42.815521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-71d6bd71-168e-4bbf-9559-61e45ffb6562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:42.815521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:42.815521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:42.815521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:42.815521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:42.815521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:42.815521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:42.815521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:42.815521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:42.815521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:42.815521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:42.815521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:42.815521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:42.815521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:42.815521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:42.816437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:42.816437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:42.816437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:42.816437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:42.816437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:42.816437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:42.816437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:42.816437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:42.816437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:42.816437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:42.816437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:42.816437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:42.816437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:42.816437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:42.816437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:42.816437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:42.816437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:42.817445 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:42.817445 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:42.817445 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:42.817445 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:42.817445 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 529/1073] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:42.822659 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cdefac-d010-498e-8730-99171f803fc6 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:42.921565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cdefac-d010-498e-8730-99171f803fc6 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:42.954863 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8387331-d140-40d5-8e5e-2bedad8d4e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:42.955520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8387331-d140-40d5-8e5e-2bedad8d4e6c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:42.955808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8387331-d140-40d5-8e5e-2bedad8d4e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:42.956084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8387331-d140-40d5-8e5e-2bedad8d4e6c None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:42.956796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8387331-d140-40d5-8e5e-2bedad8d4e6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:42.957102 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8387331-d140-40d5-8e5e-2bedad8d4e6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:42.958492 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8387331-d140-40d5-8e5e-2bedad8d4e6c None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:42.969309 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 545/1074] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:42.975386 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6bd71-168e-4bbf-9559-61e45ffb6562 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:43.005548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:43.080320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6bd71-168e-4bbf-9559-61e45ffb6562 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:43.120860 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31dc19b-0dcb-4f54-9c8f-b1b825d5abb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:43.121764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31dc19b-0dcb-4f54-9c8f-b1b825d5abb4 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:43.122109 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31dc19b-0dcb-4f54-9c8f-b1b825d5abb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:43.122483 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31dc19b-0dcb-4f54-9c8f-b1b825d5abb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:43.129610 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31dc19b-0dcb-4f54-9c8f-b1b825d5abb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:43.131146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31dc19b-0dcb-4f54-9c8f-b1b825d5abb4 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:43.145018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-f31dc19b-0dcb-4f54-9c8f-b1b825d5abb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:43.145018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:43.145018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:43.145018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:43.145018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:43.145018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:43.145018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:43.145018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:43.145018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:43.145018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:43.145018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:43.145018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:43.145018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:13:43.145018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:13:43.145018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:13:43.146053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:13:43.146053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:43.146053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:43.146053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:43.146053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:43.146053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:43.146053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:43.146053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:43.146053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:43.146053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:43.146053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:43.146053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:43.146053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:43.146053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:43.146053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:43.146053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:43.146053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:43.147241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:13:43.147241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:13:43.147241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:13:43.147241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:13:43.147241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:13:43.147241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:13:43.147241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:43.147241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:43.147982 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 530/1075] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:43.154127 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8387331-d140-40d5-8e5e-2bedad8d4e6c None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:43.179783 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:43.251879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8387331-d140-40d5-8e5e-2bedad8d4e6c None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:43.285209 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa2491-c8e6-4153-898e-7808c7358fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:43.285869 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa2491-c8e6-4153-898e-7808c7358fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:43.286149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa2491-c8e6-4153-898e-7808c7358fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:43.286457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa2491-c8e6-4153-898e-7808c7358fc7 None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:43.287239 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa2491-c8e6-4153-898e-7808c7358fc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:43.287546 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa2491-c8e6-4153-898e-7808c7358fc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:43.289286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa2491-c8e6-4153-898e-7808c7358fc7 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:43.306059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 546/1076] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:13:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:43.307496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31dc19b-0dcb-4f54-9c8f-b1b825d5abb4 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:43.395529 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:43.407032 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31dc19b-0dcb-4f54-9c8f-b1b825d5abb4 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:43.442029 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704e2d4-c848-4ece-b503-99f65d1a049f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:43.442867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704e2d4-c848-4ece-b503-99f65d1a049f None admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:43.443148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704e2d4-c848-4ece-b503-99f65d1a049f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:43.443419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704e2d4-c848-4ece-b503-99f65d1a049f None admin] PATH_INFO: `/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/a4a24d5a375945779fa626fae25ac237/roles/e2d60ba4ec674cbb8710971ec3f9bdfe` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:43.445577 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704e2d4-c848-4ece-b503-99f65d1a049f None admin] RBAC: Authorizing `identity:create_grant(project_id=cdae1cb71dd449b8a18062b7b52889e4, user_id=a4a24d5a375945779fa626fae25ac237, role_id=e2d60ba4ec674cbb8710971ec3f9bdfe)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:43.447018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704e2d4-c848-4ece-b503-99f65d1a049f None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:43.552213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 531/1077] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:13:43 2022] PUT /identity/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/a4a24d5a375945779fa626fae25ac237/roles/e2d60ba4ec674cbb8710971ec3f9bdfe => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:13:43.677940 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20131fc0-1f83-496a-9acc-3bb0d271f3b0 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:43.678495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20131fc0-1f83-496a-9acc-3bb0d271f3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:43.678581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20131fc0-1f83-496a-9acc-3bb0d271f3b0 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:43.679906 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 547/1078] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:43.685448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1dc6c-d5d0-4248-b116-47941da7d55a None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:43.685740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1dc6c-d5d0-4248-b116-47941da7d55a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:43.686020 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1dc6c-d5d0-4248-b116-47941da7d55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:43.704069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-b9d1dc6c-d5d0-4248-b116-47941da7d55a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:43.705735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b9d1dc6c-d5d0-4248-b116-47941da7d55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:43.771134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 532/1079] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:43.793472 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20131fc0-1f83-496a-9acc-3bb0d271f3b0 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:43.877341 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:43.893615 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20131fc0-1f83-496a-9acc-3bb0d271f3b0 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:43.927041 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f4216-3e54-44a3-bd8d-585e15b68c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:43.927832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f4216-3e54-44a3-bd8d-585e15b68c78 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:43.928113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f4216-3e54-44a3-bd8d-585e15b68c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:43.928387 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f4216-3e54-44a3-bd8d-585e15b68c78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:43.938305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f4216-3e54-44a3-bd8d-585e15b68c78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:13:43.946421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-847f4216-3e54-44a3-bd8d-585e15b68c78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:13:43.946421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:43.946421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:43.946421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:43.946421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:43.946421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:43.946421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:43.946421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:43.946421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:43.946421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:43.946421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:43.946421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:43.946421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:13:43.946421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:13:43.946421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:13:43.946421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:13:43.947383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:13:43.947383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:13:43.947383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:13:43.947383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:13:43.947383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:43.947383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:43.947383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:43.947383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:43.947383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:43.947383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:43.947383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:43.947383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:43.947383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:43.947383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:43.947383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:43.947383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:43.948461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:43.948461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:43.948461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:43.948461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:43.948461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:13:43.948461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:13:43.948461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:13:43.948461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:13:43.948461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:13:43.948461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:13:43.948461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:13:43.948461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:43.948461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 548/1080] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:13:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:43.953517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1dc6c-d5d0-4248-b116-47941da7d55a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:44.028421 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:44.049212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1dc6c-d5d0-4248-b116-47941da7d55a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:44.082308 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d15ce3-d104-40a2-86ef-4337f25d74ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:44.082982 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d15ce3-d104-40a2-86ef-4337f25d74ff None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:44.083271 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d15ce3-d104-40a2-86ef-4337f25d74ff None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:44.083544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d15ce3-d104-40a2-86ef-4337f25d74ff None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:44.084289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d15ce3-d104-40a2-86ef-4337f25d74ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:44.084595 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d15ce3-d104-40a2-86ef-4337f25d74ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:44.086012 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d15ce3-d104-40a2-86ef-4337f25d74ff None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:44.096317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 533/1081] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:13:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:44.102254 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f4216-3e54-44a3-bd8d-585e15b68c78 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:44.200381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f4216-3e54-44a3-bd8d-585e15b68c78 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:44.232812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8196d-5e49-4cd8-93ab-f5aa3cf97960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:44.233671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8196d-5e49-4cd8-93ab-f5aa3cf97960 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:44.233949 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8196d-5e49-4cd8-93ab-f5aa3cf97960 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:44.234220 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8196d-5e49-4cd8-93ab-f5aa3cf97960 None admin] PATH_INFO: `/v3/users/0c6582746c484d378e0dce098f10456b` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:44.235423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8196d-5e49-4cd8-93ab-f5aa3cf97960 None admin] RBAC: Authorizing `identity:get_user(user_id=0c6582746c484d378e0dce098f10456b)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:44.236966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8196d-5e49-4cd8-93ab-f5aa3cf97960 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:44.238607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 549/1082] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:13:44 2022] GET /identity/v3/users/0c6582746c484d378e0dce098f10456b => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:44.244467 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d15ce3-d104-40a2-86ef-4337f25d74ff None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:44.342786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d15ce3-d104-40a2-86ef-4337f25d74ff None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:44.375660 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3911f-34a4-434f-8a1d-474112e55199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:44.378041 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3911f-34a4-434f-8a1d-474112e55199 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:44.378041 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3911f-34a4-434f-8a1d-474112e55199 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:44.378041 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3911f-34a4-434f-8a1d-474112e55199 None admin] PATH_INFO: `/v3/projects/af45393e1609453aa5bbbc460146d8e5` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:44.379161 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3911f-34a4-434f-8a1d-474112e55199 None admin] RBAC: Authorizing `identity:get_project(project_id=af45393e1609453aa5bbbc460146d8e5)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:44.380581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3911f-34a4-434f-8a1d-474112e55199 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:44.387048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 534/1083] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:13:44 2022] GET /identity/v3/projects/af45393e1609453aa5bbbc460146d8e5 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:44.388484 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8196d-5e49-4cd8-93ab-f5aa3cf97960 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:44.403654 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:44.485231 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8196d-5e49-4cd8-93ab-f5aa3cf97960 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:44.518446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a666d0-aac5-41ee-be63-f03cb1422d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:44.519331 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a666d0-aac5-41ee-be63-f03cb1422d0f None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:44.519612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a666d0-aac5-41ee-be63-f03cb1422d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:44.519889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a666d0-aac5-41ee-be63-f03cb1422d0f None admin] PATH_INFO: `/v3/projects/af45393e1609453aa5bbbc460146d8e5/users/0c6582746c484d378e0dce098f10456b/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:44.522106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a666d0-aac5-41ee-be63-f03cb1422d0f None admin] RBAC: Authorizing `identity:create_grant(project_id=af45393e1609453aa5bbbc460146d8e5, user_id=0c6582746c484d378e0dce098f10456b, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:44.523575 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a666d0-aac5-41ee-be63-f03cb1422d0f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:44.579725 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 550/1084] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:13:44 2022] PUT /identity/v3/projects/af45393e1609453aa5bbbc460146d8e5/users/0c6582746c484d378e0dce098f10456b/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:13:45.577894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7565ccc-6b03-4e3e-9c23-350acf5430c2 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:45.578458 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7565ccc-6b03-4e3e-9c23-350acf5430c2 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:45.578548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7565ccc-6b03-4e3e-9c23-350acf5430c2 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:45.580372 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 535/1085] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:45.585425 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d56f71-1f6c-45cd-bff1-bae4da75ecaa None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:45.585819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d56f71-1f6c-45cd-bff1-bae4da75ecaa None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:45.586111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d56f71-1f6c-45cd-bff1-bae4da75ecaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:45.603091 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-79d56f71-1f6c-45cd-bff1-bae4da75ecaa None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:45.604275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-79d56f71-1f6c-45cd-bff1-bae4da75ecaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:45.669473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 551/1086] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:45.693373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7565ccc-6b03-4e3e-9c23-350acf5430c2 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:45.705812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:45.794158 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7565ccc-6b03-4e3e-9c23-350acf5430c2 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:45.826926 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785405b-ee45-40e6-a34b-3dd428ec5d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:45.827725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785405b-ee45-40e6-a34b-3dd428ec5d87 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:45.828053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785405b-ee45-40e6-a34b-3dd428ec5d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:45.828275 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785405b-ee45-40e6-a34b-3dd428ec5d87 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:45.838724 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785405b-ee45-40e6-a34b-3dd428ec5d87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:13:45.846838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-2785405b-ee45-40e6-a34b-3dd428ec5d87 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:13:45.846838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:45.846838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:45.846838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:45.846838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:45.846838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:45.846838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:45.846838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:45.846838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:45.846838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:45.846838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:45.846838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:45.846838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:13:45.846838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:13:45.846838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:13:45.846838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:13:45.847820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:13:45.847820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:13:45.847820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:13:45.847820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:13:45.847820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:45.847820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:45.847820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:45.847820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:45.847820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:45.847820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:45.847820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:45.847820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:45.847820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:45.847820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:45.847820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:45.847820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:45.848734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:45.848734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:45.848734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:45.848734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:45.848734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:13:45.848734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:13:45.848734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:13:45.848734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:13:45.848734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:13:45.848734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:13:45.848734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 15:13:45.848734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:45.848734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 536/1087] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Jul 20 15:13:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:45.853910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d56f71-1f6c-45cd-bff1-bae4da75ecaa None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:45.856202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:45.950187 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d56f71-1f6c-45cd-bff1-bae4da75ecaa None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:45.982412 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf85355-3746-4c82-827f-de5bcb52829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:45.983066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf85355-3746-4c82-827f-de5bcb52829c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:45.983346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf85355-3746-4c82-827f-de5bcb52829c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:45.983619 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf85355-3746-4c82-827f-de5bcb52829c None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:45.984351 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf85355-3746-4c82-827f-de5bcb52829c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:45.984652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf85355-3746-4c82-827f-de5bcb52829c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:45.986559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf85355-3746-4c82-827f-de5bcb52829c None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:45.994112 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 552/1088] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Jul 20 15:13:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:45.999583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785405b-ee45-40e6-a34b-3dd428ec5d87 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:46.063546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:46.098656 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785405b-ee45-40e6-a34b-3dd428ec5d87 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:46.132134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad54c51-ac21-44c5-b6d3-eb442a703bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:46.132916 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad54c51-ac21-44c5-b6d3-eb442a703bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:46.133198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad54c51-ac21-44c5-b6d3-eb442a703bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:46.133473 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad54c51-ac21-44c5-b6d3-eb442a703bc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:46.143833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad54c51-ac21-44c5-b6d3-eb442a703bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:46.145254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad54c51-ac21-44c5-b6d3-eb442a703bc2 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:46.161725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-7ad54c51-ac21-44c5-b6d3-eb442a703bc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:46.161725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:46.161725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:46.161725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:46.161725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:46.161725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:46.161725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:46.161725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:46.161725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:46.161725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:46.161725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:46.161725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:46.161725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:46.161725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:46.161725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:46.162987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:46.162987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:46.162987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:46.162987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:46.162987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:46.162987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:46.162987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:46.162987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:46.162987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:46.162987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:46.162987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:46.162987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:46.162987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:46.162987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:46.162987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:46.162987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:46.162987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:46.164138 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:46.164138 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:46.164138 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:46.164138 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:46.166434 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 537/1089] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:46.168673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf85355-3746-4c82-827f-de5bcb52829c None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:46.228968 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:46.270097 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf85355-3746-4c82-827f-de5bcb52829c None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:46.305380 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5466a49-683b-4136-b614-852bf00c1aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:46.306052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5466a49-683b-4136-b614-852bf00c1aaa None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:46.306420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5466a49-683b-4136-b614-852bf00c1aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:46.306662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5466a49-683b-4136-b614-852bf00c1aaa None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:46.307388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5466a49-683b-4136-b614-852bf00c1aaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:46.307698 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5466a49-683b-4136-b614-852bf00c1aaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:46.309083 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5466a49-683b-4136-b614-852bf00c1aaa None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:46.325901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 553/1090] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:46.327246 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad54c51-ac21-44c5-b6d3-eb442a703bc2 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:46.432630 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad54c51-ac21-44c5-b6d3-eb442a703bc2 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:46.451476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:46.465536 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a8e40-934c-49ee-a200-62ec985b2351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:46.466511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a8e40-934c-49ee-a200-62ec985b2351 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:46.466797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a8e40-934c-49ee-a200-62ec985b2351 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:46.467076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a8e40-934c-49ee-a200-62ec985b2351 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:46.475069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a8e40-934c-49ee-a200-62ec985b2351 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:46.476511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a8e40-934c-49ee-a200-62ec985b2351 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:46.493535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-1d4a8e40-934c-49ee-a200-62ec985b2351 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:13:46.493535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:46.493535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:46.493535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:46.493535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:46.493535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:46.493535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:46.493535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:46.493535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:46.493535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:46.493535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:46.493535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:46.493535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:13:46.493535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:13:46.493535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:13:46.493535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:46.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:46.501100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:46.501100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:46.501100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:13:46.501100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:13:46.501100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:13:46.501100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:13:46.501100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:13:46.501100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:13:46.501100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 15:13:46.501100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:46.501100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 538/1091] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Jul 20 15:13:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:13:46.501833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5466a49-683b-4136-b614-852bf00c1aaa None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:46.600116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5466a49-683b-4136-b614-852bf00c1aaa None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:46.614976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3b79d-84f8-43e3-8174-69901385a128 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:46.615431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3b79d-84f8-43e3-8174-69901385a128 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:46.615524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3b79d-84f8-43e3-8174-69901385a128 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:46.616905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 539/1092] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:13:46.621782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684acee-3492-4b2d-a440-bd4e56a72ec5 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:46.622070 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684acee-3492-4b2d-a440-bd4e56a72ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:46.622398 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684acee-3492-4b2d-a440-bd4e56a72ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:46.624799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:46.635866 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004c165-a469-4bbf-9ccd-295551c7b401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:46.636539 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004c165-a469-4bbf-9ccd-295551c7b401 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:46.637107 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004c165-a469-4bbf-9ccd-295551c7b401 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:46.637107 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004c165-a469-4bbf-9ccd-295551c7b401 None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:46.638084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004c165-a469-4bbf-9ccd-295551c7b401 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:46.638439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004c165-a469-4bbf-9ccd-295551c7b401 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:46.639284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-b684acee-3492-4b2d-a440-bd4e56a72ec5 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:46.639874 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004c165-a469-4bbf-9ccd-295551c7b401 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:46.640509 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b684acee-3492-4b2d-a440-bd4e56a72ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:46.657925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 554/1093] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Jul 20 15:13:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:46.663336 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004c165-a469-4bbf-9ccd-295551c7b401 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:46.707647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 540/1094] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 15:13:46.731337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684acee-3492-4b2d-a440-bd4e56a72ec5 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:46.759329 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004c165-a469-4bbf-9ccd-295551c7b401 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:46.797639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a1808-6977-4b8d-bc1b-f45f314d08d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:46.798345 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a1808-6977-4b8d-bc1b-f45f314d08d4 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:46.798630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a1808-6977-4b8d-bc1b-f45f314d08d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:46.798905 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a1808-6977-4b8d-bc1b-f45f314d08d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:46.832072 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684acee-3492-4b2d-a440-bd4e56a72ec5 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:46.844239 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:46.851209 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:46.870116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a1808-6977-4b8d-bc1b-f45f314d08d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:46.877471 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a1808-6977-4b8d-bc1b-f45f314d08d4 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:46.877601 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6714e06-75e5-402d-bf88-903abe6a998e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:46.877601 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6714e06-75e5-402d-bf88-903abe6a998e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:46.877890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6714e06-75e5-402d-bf88-903abe6a998e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:46.878171 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6714e06-75e5-402d-bf88-903abe6a998e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:46.890493 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6714e06-75e5-402d-bf88-903abe6a998e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:13:46.899559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-e6714e06-75e5-402d-bf88-903abe6a998e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:13:46.899559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:46.899559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:46.899559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:46.899559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:46.899559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:46.899559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:46.899559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:46.899559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:46.899559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:46.899559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:46.899559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:46.899559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:13:46.899559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:13:46.899559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:13:46.899559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:13:46.900808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:13:46.900808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:13:46.900808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:13:46.900808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:13:46.900808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:46.900808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:46.900808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:46.900808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:46.900808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:46.900808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:46.900808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:46.900808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:46.900808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:46.900808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:46.900808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:46.900808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:46.902055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:46.902055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:46.902055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:46.902055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:46.902055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:13:46.902055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:13:46.902055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:13:46.902055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:13:46.902055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:13:46.902055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:13:46.902055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:13:46.902055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:13:46.902055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 541/1095] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:13:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:46.906732 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6714e06-75e5-402d-bf88-903abe6a998e None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:46.972724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 555/1096] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:13:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 15:13:46.978287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a1808-6977-4b8d-bc1b-f45f314d08d4 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:47.009694 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6714e06-75e5-402d-bf88-903abe6a998e None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:47.042966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f520772-8ea0-4676-8ffc-05b2200c308e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:47.043623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f520772-8ea0-4676-8ffc-05b2200c308e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:47.043914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f520772-8ea0-4676-8ffc-05b2200c308e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:47.044195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f520772-8ea0-4676-8ffc-05b2200c308e None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:47.044942 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f520772-8ea0-4676-8ffc-05b2200c308e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:47.045254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f520772-8ea0-4676-8ffc-05b2200c308e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:47.046654 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f520772-8ea0-4676-8ffc-05b2200c308e None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:47.054553 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 542/1097] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:13:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:47.055853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:47.060417 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f520772-8ea0-4676-8ffc-05b2200c308e None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:47.073029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a1808-6977-4b8d-bc1b-f45f314d08d4 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:47.092011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:47.111696 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7385f-0e62-468f-9c3a-2462b183f816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:47.112576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7385f-0e62-468f-9c3a-2462b183f816 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:47.112865 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7385f-0e62-468f-9c3a-2462b183f816 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:47.113140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7385f-0e62-468f-9c3a-2462b183f816 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:47.123188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7385f-0e62-468f-9c3a-2462b183f816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:47.124634 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7385f-0e62-468f-9c3a-2462b183f816 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:47.143551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-68d7385f-0e62-468f-9c3a-2462b183f816 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:47.143551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:47.143551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:47.143551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:47.143551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:47.143551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:47.143551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:47.143551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:47.143551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:47.143551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:47.143551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:47.143551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:47.143551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:13:47.143551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:13:47.143551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:13:47.144629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:13:47.144629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:47.144629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:47.144629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:47.144629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:47.144629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:47.144629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:47.144629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:47.144629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:47.144629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:47.144629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:47.144629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:47.144629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:47.144629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:47.144629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:47.144629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:47.144629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:47.145582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:13:47.145582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:13:47.145582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:13:47.145582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:47.145582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 556/1098] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:47.150461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7385f-0e62-468f-9c3a-2462b183f816 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:47.164493 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f520772-8ea0-4676-8ffc-05b2200c308e None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:47.198365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe049df0-94a1-4d70-a407-2c5add4af0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:47.199237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe049df0-94a1-4d70-a407-2c5add4af0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:47.199522 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe049df0-94a1-4d70-a407-2c5add4af0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:47.199797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe049df0-94a1-4d70-a407-2c5add4af0f8 None admin] PATH_INFO: `/v3/users/0c6582746c484d378e0dce098f10456b` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:47.200996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe049df0-94a1-4d70-a407-2c5add4af0f8 None admin] RBAC: Authorizing `identity:get_user(user_id=0c6582746c484d378e0dce098f10456b)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:47.202553 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe049df0-94a1-4d70-a407-2c5add4af0f8 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:47.204732 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 543/1099] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:13:47 2022] GET /identity/v3/users/0c6582746c484d378e0dce098f10456b => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:47.210399 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe049df0-94a1-4d70-a407-2c5add4af0f8 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:47.246794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7385f-0e62-468f-9c3a-2462b183f816 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:47.281405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f6e6b-467b-435a-a2e4-c5ec0003dcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:47.282070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f6e6b-467b-435a-a2e4-c5ec0003dcfa None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:47.282384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f6e6b-467b-435a-a2e4-c5ec0003dcfa None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:47.282663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f6e6b-467b-435a-a2e4-c5ec0003dcfa None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:47.283379 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f6e6b-467b-435a-a2e4-c5ec0003dcfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:47.283683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f6e6b-467b-435a-a2e4-c5ec0003dcfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:47.285052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f6e6b-467b-435a-a2e4-c5ec0003dcfa None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:47.286481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:47.296849 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 557/1100] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:47.302441 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f6e6b-467b-435a-a2e4-c5ec0003dcfa None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:47.305818 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:47.306916 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe049df0-94a1-4d70-a407-2c5add4af0f8 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:47.340138 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e688e7-680d-406f-999d-2d959d91bbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:47.340783 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e688e7-680d-406f-999d-2d959d91bbcb None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:47.341074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e688e7-680d-406f-999d-2d959d91bbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:47.341353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e688e7-680d-406f-999d-2d959d91bbcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:47.400763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f6e6b-467b-435a-a2e4-c5ec0003dcfa None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:47.408132 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e688e7-680d-406f-999d-2d959d91bbcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:47.409618 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e688e7-680d-406f-999d-2d959d91bbcb None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:47.435456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af704504-f7cf-46e6-87b0-91024e5a61b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:47.436259 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af704504-f7cf-46e6-87b0-91024e5a61b2 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:47.436544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af704504-f7cf-46e6-87b0-91024e5a61b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:47.436827 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af704504-f7cf-46e6-87b0-91024e5a61b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:47.443651 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af704504-f7cf-46e6-87b0-91024e5a61b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:47.445119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af704504-f7cf-46e6-87b0-91024e5a61b2 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:47.459204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-af704504-f7cf-46e6-87b0-91024e5a61b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:47.459204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:13:47.459204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:13:47.459204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:13:47.459204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:13:47.459204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:13:47.459204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:13:47.459204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:13:47.459204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:13:47.459204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:13:47.459204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:13:47.459204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:13:47.459204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:13:47.459204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:13:47.459204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:13:47.460335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:13:47.460335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:13:47.460335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:13:47.460335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:13:47.460335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:13:47.460335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:13:47.460335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:13:47.460335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:13:47.460335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:13:47.460335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:13:47.460335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:13:47.460335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:13:47.460335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:13:47.460335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:13:47.460335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:13:47.460335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:13:47.460335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:13:47.461289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:13:47.461289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:13:47.461289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:13:47.461289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:13:47.461289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:13:47.461289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:13:47.461289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:13:47.461289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:13:47.461289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 558/1101] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:13:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:13:47.466281 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af704504-f7cf-46e6-87b0-91024e5a61b2 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:47.507198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:47.511229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 544/1102] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:13:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 15:13:47.517520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e688e7-680d-406f-999d-2d959d91bbcb None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:47.534625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:47.567116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af704504-f7cf-46e6-87b0-91024e5a61b2 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:47.601061 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e1f0c-9a1e-47cc-9052-28483703d362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:47.601724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e1f0c-9a1e-47cc-9052-28483703d362 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:47.602010 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e1f0c-9a1e-47cc-9052-28483703d362 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:47.602316 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e1f0c-9a1e-47cc-9052-28483703d362 None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:47.603133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e1f0c-9a1e-47cc-9052-28483703d362 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:47.603471 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e1f0c-9a1e-47cc-9052-28483703d362 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:47.604913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e1f0c-9a1e-47cc-9052-28483703d362 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:47.613782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e688e7-680d-406f-999d-2d959d91bbcb None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:47.616603 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 559/1103] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:13:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:47.622515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e1f0c-9a1e-47cc-9052-28483703d362 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:47.651997 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485087ca-f0f9-4ab0-9711-9aea5d7e0c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:47.652854 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485087ca-f0f9-4ab0-9711-9aea5d7e0c16 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:47.653132 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485087ca-f0f9-4ab0-9711-9aea5d7e0c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:47.653408 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485087ca-f0f9-4ab0-9711-9aea5d7e0c16 None admin] PATH_INFO: `/v3/projects/af45393e1609453aa5bbbc460146d8e5` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:47.654696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485087ca-f0f9-4ab0-9711-9aea5d7e0c16 None admin] RBAC: Authorizing `identity:get_project(project_id=af45393e1609453aa5bbbc460146d8e5)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:47.656120 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485087ca-f0f9-4ab0-9711-9aea5d7e0c16 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:47.658481 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 545/1104] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:13:47 2022] GET /identity/v3/projects/af45393e1609453aa5bbbc460146d8e5 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:47.664502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485087ca-f0f9-4ab0-9711-9aea5d7e0c16 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:47.719771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e1f0c-9a1e-47cc-9052-28483703d362 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:47.729081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:47.734131 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:47.753659 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5561f98-ca0c-4978-8769-ad0e24404459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:47.754341 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5561f98-ca0c-4978-8769-ad0e24404459 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:47.754753 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5561f98-ca0c-4978-8769-ad0e24404459 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:47.755033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5561f98-ca0c-4978-8769-ad0e24404459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:47.755856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5561f98-ca0c-4978-8769-ad0e24404459 None admin] RBAC: Adding query filter params (role.id=e2d60ba4ec674cbb8710971ec3f9bdfe, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=a4a24d5a375945779fa626fae25ac237) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:47.756167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5561f98-ca0c-4978-8769-ad0e24404459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:47.757639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5561f98-ca0c-4978-8769-ad0e24404459 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:47.761375 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485087ca-f0f9-4ab0-9711-9aea5d7e0c16 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:47.763706 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 560/1105] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:13:47 2022] GET /identity/v3/role_assignments?user.id=a4a24d5a375945779fa626fae25ac237&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=e2d60ba4ec674cbb8710971ec3f9bdfe => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:47.798532 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf2a30-470b-4abb-b20f-680c7e326993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:47.799198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf2a30-470b-4abb-b20f-680c7e326993 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:47.799478 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf2a30-470b-4abb-b20f-680c7e326993 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:47.799753 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf2a30-470b-4abb-b20f-680c7e326993 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:47.800596 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf2a30-470b-4abb-b20f-680c7e326993 None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=af45393e1609453aa5bbbc460146d8e5, user.id=0c6582746c484d378e0dce098f10456b) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:47.800907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf2a30-470b-4abb-b20f-680c7e326993 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:47.802414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf2a30-470b-4abb-b20f-680c7e326993 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:47.809130 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 546/1106] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:13:47 2022] GET /identity/v3/role_assignments?user.id=0c6582746c484d378e0dce098f10456b&scope.project.id=af45393e1609453aa5bbbc460146d8e5&role.id=25516be1485c49538544ea8a596e7d17 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:13:49.868276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85718dbc-faed-4291-aaef-a10b6c5adb07 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:49.868927 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85718dbc-faed-4291-aaef-a10b6c5adb07 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:49.868927 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85718dbc-faed-4291-aaef-a10b6c5adb07 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:49.870187 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 561/1107] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:49.875384 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76eecc-16f7-4ed8-8fef-9f6bf5735bcd None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:49.876093 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76eecc-16f7-4ed8-8fef-9f6bf5735bcd None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:49.876279 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76eecc-16f7-4ed8-8fef-9f6bf5735bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:49.892975 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-fe76eecc-16f7-4ed8-8fef-9f6bf5735bcd None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:49.894154 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-fe76eecc-16f7-4ed8-8fef-9f6bf5735bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:49.957463 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 547/1108] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:49.980905 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85718dbc-faed-4291-aaef-a10b6c5adb07 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:50.078223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85718dbc-faed-4291-aaef-a10b6c5adb07 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:50.112807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f04ed6-85e8-41a5-be7f-73f46e75cfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:50.113589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f04ed6-85e8-41a5-be7f-73f46e75cfe1 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:50.113868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f04ed6-85e8-41a5-be7f-73f46e75cfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:50.114146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f04ed6-85e8-41a5-be7f-73f46e75cfe1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:50.115436 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f04ed6-85e8-41a5-be7f-73f46e75cfe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:50.116857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f04ed6-85e8-41a5-be7f-73f46e75cfe1 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:50.118508 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 562/1109] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:13:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:50.124146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76eecc-16f7-4ed8-8fef-9f6bf5735bcd None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:50.182094 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:50.219597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76eecc-16f7-4ed8-8fef-9f6bf5735bcd None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:50.251781 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5681d2-6bb8-4093-a5e7-1abf7eec7804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:50.252429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5681d2-6bb8-4093-a5e7-1abf7eec7804 None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:50.252709 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5681d2-6bb8-4093-a5e7-1abf7eec7804 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:50.252988 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5681d2-6bb8-4093-a5e7-1abf7eec7804 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:50.253695 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5681d2-6bb8-4093-a5e7-1abf7eec7804 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:50.255136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5681d2-6bb8-4093-a5e7-1abf7eec7804 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:50.351416 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 548/1110] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:13:50 2022] POST /identity/v3/users => generated 286 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:13:52.398117 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394db23c-3968-4e39-abb3-8e02ce300f20 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:52.398664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394db23c-3968-4e39-abb3-8e02ce300f20 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:52.398664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394db23c-3968-4e39-abb3-8e02ce300f20 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:52.399989 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 563/1111] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:52.405035 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75166122-e2c8-48f5-afe5-2c01f13b2432 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:52.405457 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75166122-e2c8-48f5-afe5-2c01f13b2432 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:52.405588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75166122-e2c8-48f5-afe5-2c01f13b2432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:52.422301 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-75166122-e2c8-48f5-afe5-2c01f13b2432 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:52.424066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-75166122-e2c8-48f5-afe5-2c01f13b2432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:52.487456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 549/1112] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:52.510838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394db23c-3968-4e39-abb3-8e02ce300f20 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:52.557795 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:52.609499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394db23c-3968-4e39-abb3-8e02ce300f20 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:52.641808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d9180-1e98-4c83-a23d-f2a237ceecc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:52.642681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d9180-1e98-4c83-a23d-f2a237ceecc2 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:52.642963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d9180-1e98-4c83-a23d-f2a237ceecc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:52.643234 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d9180-1e98-4c83-a23d-f2a237ceecc2 None admin] PATH_INFO: `/v3/roles/2da8d544efe44ce6a41160d5759be402` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:52.644699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d9180-1e98-4c83-a23d-f2a237ceecc2 None admin] RBAC: Authorizing `identity:get_role(role_id=2da8d544efe44ce6a41160d5759be402)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:52.646059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d9180-1e98-4c83-a23d-f2a237ceecc2 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:52.648321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 564/1113] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:13:52 2022] GET /identity/v3/roles/2da8d544efe44ce6a41160d5759be402 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:52.654202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75166122-e2c8-48f5-afe5-2c01f13b2432 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:52.701228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:52.749992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75166122-e2c8-48f5-afe5-2c01f13b2432 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:52.783656 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd9a1f-2341-42ca-8eb2-5a945bece84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:52.784518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd9a1f-2341-42ca-8eb2-5a945bece84e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:52.784802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd9a1f-2341-42ca-8eb2-5a945bece84e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:52.785078 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd9a1f-2341-42ca-8eb2-5a945bece84e None admin] PATH_INFO: `/v3/users/39a3711f8c0440a88728386c48332f7e` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:52.798392 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd9a1f-2341-42ca-8eb2-5a945bece84e None admin] RBAC: Authorizing `identity:get_user(user_id=39a3711f8c0440a88728386c48332f7e)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:52.799852 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd9a1f-2341-42ca-8eb2-5a945bece84e None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:52.801475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 550/1114] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:13:52 2022] GET /identity/v3/users/39a3711f8c0440a88728386c48332f7e => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:52.807251 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d9180-1e98-4c83-a23d-f2a237ceecc2 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:52.902608 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d9180-1e98-4c83-a23d-f2a237ceecc2 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:52.915793 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:52.935179 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703303af-a816-4c29-bc3d-79c1c646144d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:52.935815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703303af-a816-4c29-bc3d-79c1c646144d None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:52.936100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703303af-a816-4c29-bc3d-79c1c646144d None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:52.936376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703303af-a816-4c29-bc3d-79c1c646144d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:53.000380 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703303af-a816-4c29-bc3d-79c1c646144d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:53.002646 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703303af-a816-4c29-bc3d-79c1c646144d None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:53.095654 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 565/1115] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:13:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:53.101451 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd9a1f-2341-42ca-8eb2-5a945bece84e None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:53.197620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd9a1f-2341-42ca-8eb2-5a945bece84e None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:53.204588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:53.230270 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df59e1b-2ff3-4897-901f-0021c2eefc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:53.231133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df59e1b-2ff3-4897-901f-0021c2eefc98 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:53.231415 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df59e1b-2ff3-4897-901f-0021c2eefc98 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:53.231688 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df59e1b-2ff3-4897-901f-0021c2eefc98 None admin] PATH_INFO: `/v3/projects/af45393e1609453aa5bbbc460146d8e5` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:53.232849 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df59e1b-2ff3-4897-901f-0021c2eefc98 None admin] RBAC: Authorizing `identity:get_project(project_id=af45393e1609453aa5bbbc460146d8e5)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:53.234305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df59e1b-2ff3-4897-901f-0021c2eefc98 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:53.236533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 551/1116] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:13:53 2022] GET /identity/v3/projects/af45393e1609453aa5bbbc460146d8e5 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:53.242532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703303af-a816-4c29-bc3d-79c1c646144d None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:53.257185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:53.336769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703303af-a816-4c29-bc3d-79c1c646144d None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:53.370711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bea0c-f334-4c04-be65-f129f7a83bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:53.371356 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bea0c-f334-4c04-be65-f129f7a83bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:53.371643 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bea0c-f334-4c04-be65-f129f7a83bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:53.371920 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bea0c-f334-4c04-be65-f129f7a83bd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:53.372742 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961bea0c-f334-4c04-be65-f129f7a83bd6 None admin] RBAC: Adding query filter params (role.id=2da8d544efe44ce6a41160d5759be402, scope.project.id=af45393e1609453aa5bbbc460146d8e5, user.id=39a3711f8c0440a88728386c48332f7e) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:53.373055 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961bea0c-f334-4c04-be65-f129f7a83bd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:53.374526 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961bea0c-f334-4c04-be65-f129f7a83bd6 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:53.380224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 566/1117] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:13:53 2022] GET /identity/v3/role_assignments?user.id=39a3711f8c0440a88728386c48332f7e&scope.project.id=af45393e1609453aa5bbbc460146d8e5&role.id=2da8d544efe44ce6a41160d5759be402 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:55.378873 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e3816-5da7-4764-b0ac-ce26203c172e None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:55.379392 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e3816-5da7-4764-b0ac-ce26203c172e None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:55.379392 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e3816-5da7-4764-b0ac-ce26203c172e None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:55.380713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 552/1118] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:55.385795 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd490352-e2b3-48a7-993c-78db016fd9c1 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:55.386186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd490352-e2b3-48a7-993c-78db016fd9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:55.386379 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd490352-e2b3-48a7-993c-78db016fd9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:55.403154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-cd490352-e2b3-48a7-993c-78db016fd9c1 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:55.404339 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-cd490352-e2b3-48a7-993c-78db016fd9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:55.467487 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 567/1119] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:55.489805 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e3816-5da7-4764-b0ac-ce26203c172e None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:55.590697 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e3816-5da7-4764-b0ac-ce26203c172e None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:55.623031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf3691-704b-4ba7-9c80-f223f45ab2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:55.623871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf3691-704b-4ba7-9c80-f223f45ab2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:55.624150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf3691-704b-4ba7-9c80-f223f45ab2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:55.624422 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf3691-704b-4ba7-9c80-f223f45ab2c6 None admin] PATH_INFO: `/v3/roles/2da8d544efe44ce6a41160d5759be402` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:55.626688 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf3691-704b-4ba7-9c80-f223f45ab2c6 None admin] RBAC: Authorizing `identity:get_role(role_id=2da8d544efe44ce6a41160d5759be402)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:55.628035 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf3691-704b-4ba7-9c80-f223f45ab2c6 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:55.629433 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 553/1120] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:13:55 2022] GET /identity/v3/roles/2da8d544efe44ce6a41160d5759be402 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:55.635213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd490352-e2b3-48a7-993c-78db016fd9c1 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:55.645709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:55.730125 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd490352-e2b3-48a7-993c-78db016fd9c1 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:55.764090 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddd5ad-8518-4968-9c64-c2d003c0e2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:55.764943 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddd5ad-8518-4968-9c64-c2d003c0e2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:55.765223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddd5ad-8518-4968-9c64-c2d003c0e2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:55.765496 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddd5ad-8518-4968-9c64-c2d003c0e2d0 None admin] PATH_INFO: `/v3/users/39a3711f8c0440a88728386c48332f7e` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:55.766713 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddd5ad-8518-4968-9c64-c2d003c0e2d0 None admin] RBAC: Authorizing `identity:get_user(user_id=39a3711f8c0440a88728386c48332f7e)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:55.768143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddd5ad-8518-4968-9c64-c2d003c0e2d0 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:55.769736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 568/1121] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:13:55 2022] GET /identity/v3/users/39a3711f8c0440a88728386c48332f7e => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:55.775964 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf3691-704b-4ba7-9c80-f223f45ab2c6 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:55.817996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:55.872351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf3691-704b-4ba7-9c80-f223f45ab2c6 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:55.905256 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e9f40-1cc3-42f5-af15-24ccd7198b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:55.906045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e9f40-1cc3-42f5-af15-24ccd7198b34 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:55.906358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e9f40-1cc3-42f5-af15-24ccd7198b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:55.906639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e9f40-1cc3-42f5-af15-24ccd7198b34 None admin] PATH_INFO: `/v3/projects/af45393e1609453aa5bbbc460146d8e5` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:55.907834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e9f40-1cc3-42f5-af15-24ccd7198b34 None admin] RBAC: Authorizing `identity:get_project(project_id=af45393e1609453aa5bbbc460146d8e5)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:55.909252 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e9f40-1cc3-42f5-af15-24ccd7198b34 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:55.910955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 554/1122] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:13:55 2022] GET /identity/v3/projects/af45393e1609453aa5bbbc460146d8e5 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:55.917137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddd5ad-8518-4968-9c64-c2d003c0e2d0 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:55.990500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:56.012553 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddd5ad-8518-4968-9c64-c2d003c0e2d0 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:56.045182 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a56bd-7fd6-4005-820b-5836078dedbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:56.046051 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a56bd-7fd6-4005-820b-5836078dedbc None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:56.046356 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a56bd-7fd6-4005-820b-5836078dedbc None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:56.046637 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a56bd-7fd6-4005-820b-5836078dedbc None admin] PATH_INFO: `/v3/projects/af45393e1609453aa5bbbc460146d8e5/users/39a3711f8c0440a88728386c48332f7e/roles/2da8d544efe44ce6a41160d5759be402` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:56.049301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a56bd-7fd6-4005-820b-5836078dedbc None admin] RBAC: Authorizing `identity:create_grant(project_id=af45393e1609453aa5bbbc460146d8e5, user_id=39a3711f8c0440a88728386c48332f7e, role_id=2da8d544efe44ce6a41160d5759be402)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:56.050940 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a56bd-7fd6-4005-820b-5836078dedbc None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:56.122413 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 569/1123] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:13:55 2022] PUT /identity/v3/projects/af45393e1609453aa5bbbc460146d8e5/users/39a3711f8c0440a88728386c48332f7e/roles/2da8d544efe44ce6a41160d5759be402 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:13:58.154532 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492bf4-8a08-4755-b02e-cbc37dceb925 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:58.155021 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492bf4-8a08-4755-b02e-cbc37dceb925 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:58.155114 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492bf4-8a08-4755-b02e-cbc37dceb925 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:58.156463 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 555/1124] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:13:58.161680 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5f378-faa3-44eb-865d-7a1904cbeeee None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:58.162086 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5f378-faa3-44eb-865d-7a1904cbeeee None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:58.162236 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5f378-faa3-44eb-865d-7a1904cbeeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:58.179077 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-21c5f378-faa3-44eb-865d-7a1904cbeeee None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:13:58.180260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-21c5f378-faa3-44eb-865d-7a1904cbeeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:13:58.244311 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 570/1125] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:13:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:13:58.268021 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b492bf4-8a08-4755-b02e-cbc37dceb925 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:58.364894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b492bf4-8a08-4755-b02e-cbc37dceb925 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:58.374973 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:58.398047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54abf43-24e2-45d2-8ea5-7192af0e5901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:58.398907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54abf43-24e2-45d2-8ea5-7192af0e5901 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:58.399190 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54abf43-24e2-45d2-8ea5-7192af0e5901 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:58.399466 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54abf43-24e2-45d2-8ea5-7192af0e5901 None admin] PATH_INFO: `/v3/roles/2da8d544efe44ce6a41160d5759be402` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:58.400995 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54abf43-24e2-45d2-8ea5-7192af0e5901 None admin] RBAC: Authorizing `identity:get_role(role_id=2da8d544efe44ce6a41160d5759be402)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:58.402478 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54abf43-24e2-45d2-8ea5-7192af0e5901 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:58.403814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 556/1126] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:13:58 2022] GET /identity/v3/roles/2da8d544efe44ce6a41160d5759be402 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:58.410156 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5f378-faa3-44eb-865d-7a1904cbeeee None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:58.472147 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:58.505922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5f378-faa3-44eb-865d-7a1904cbeeee None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:58.538405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbda2ef-cba8-45c9-90cc-a6dc8f9f2585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:58.539263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbda2ef-cba8-45c9-90cc-a6dc8f9f2585 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:58.539541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbda2ef-cba8-45c9-90cc-a6dc8f9f2585 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:58.539813 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbda2ef-cba8-45c9-90cc-a6dc8f9f2585 None admin] PATH_INFO: `/v3/users/39a3711f8c0440a88728386c48332f7e` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:58.540988 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbda2ef-cba8-45c9-90cc-a6dc8f9f2585 None admin] RBAC: Authorizing `identity:get_user(user_id=39a3711f8c0440a88728386c48332f7e)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:58.542445 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbda2ef-cba8-45c9-90cc-a6dc8f9f2585 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:58.544033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 571/1127] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:13:58 2022] GET /identity/v3/users/39a3711f8c0440a88728386c48332f7e => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:58.550218 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54abf43-24e2-45d2-8ea5-7192af0e5901 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:58.644710 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54abf43-24e2-45d2-8ea5-7192af0e5901 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:58.677428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3af836-1fc2-4f26-86cc-abfcf946be4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:58.678054 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3af836-1fc2-4f26-86cc-abfcf946be4b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:58.678361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3af836-1fc2-4f26-86cc-abfcf946be4b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:58.678635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3af836-1fc2-4f26-86cc-abfcf946be4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:58.721573 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:58.742495 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3af836-1fc2-4f26-86cc-abfcf946be4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:58.744080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3af836-1fc2-4f26-86cc-abfcf946be4b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:58.837918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 557/1128] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:13:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:58.843771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbda2ef-cba8-45c9-90cc-a6dc8f9f2585 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:58.939437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbda2ef-cba8-45c9-90cc-a6dc8f9f2585 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:58.972300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f06c9-3f13-4652-9b3a-77e99061af2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:58.973143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f06c9-3f13-4652-9b3a-77e99061af2e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:58.973423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f06c9-3f13-4652-9b3a-77e99061af2e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:58.973697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f06c9-3f13-4652-9b3a-77e99061af2e None admin] PATH_INFO: `/v3/projects/af45393e1609453aa5bbbc460146d8e5` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:58.974877 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f06c9-3f13-4652-9b3a-77e99061af2e None admin] RBAC: Authorizing `identity:get_project(project_id=af45393e1609453aa5bbbc460146d8e5)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:58.976302 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f06c9-3f13-4652-9b3a-77e99061af2e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:58.977957 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 572/1129] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:13:58 2022] GET /identity/v3/projects/af45393e1609453aa5bbbc460146d8e5 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:13:58.983958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3af836-1fc2-4f26-86cc-abfcf946be4b None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:13:59.057542 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:59.077445 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3af836-1fc2-4f26-86cc-abfcf946be4b None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:13:59.114035 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268680d-d89a-4bf7-a2c8-4dbbfe3ca900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:13:59.114729 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268680d-d89a-4bf7-a2c8-4dbbfe3ca900 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:13:59.115042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268680d-d89a-4bf7-a2c8-4dbbfe3ca900 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:13:59.115349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268680d-d89a-4bf7-a2c8-4dbbfe3ca900 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:13:59.116174 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268680d-d89a-4bf7-a2c8-4dbbfe3ca900 None admin] RBAC: Adding query filter params (role.id=2da8d544efe44ce6a41160d5759be402, scope.project.id=af45393e1609453aa5bbbc460146d8e5, user.id=39a3711f8c0440a88728386c48332f7e) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:13:59.116490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268680d-d89a-4bf7-a2c8-4dbbfe3ca900 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:13:59.117913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268680d-d89a-4bf7-a2c8-4dbbfe3ca900 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:13:59.123611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 558/1130] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:13:58 2022] GET /identity/v3/role_assignments?user.id=39a3711f8c0440a88728386c48332f7e&scope.project.id=af45393e1609453aa5bbbc460146d8e5&role.id=2da8d544efe44ce6a41160d5759be402 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:01.168074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c133458-9e49-44bf-b67e-89cc3243c383 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:01.168584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c133458-9e49-44bf-b67e-89cc3243c383 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:01.168678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c133458-9e49-44bf-b67e-89cc3243c383 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:01.169984 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 573/1131] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:14:01.175147 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b4db5-4971-43c8-9cd7-85e229ce4e4f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:01.175540 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b4db5-4971-43c8-9cd7-85e229ce4e4f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:01.175709 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b4db5-4971-43c8-9cd7-85e229ce4e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:01.192054 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-df2b4db5-4971-43c8-9cd7-85e229ce4e4f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:14:01.193235 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-df2b4db5-4971-43c8-9cd7-85e229ce4e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:14:01.256735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 559/1132] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:14:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:14:01.279832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c133458-9e49-44bf-b67e-89cc3243c383 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:01.283329 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:01.380012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c133458-9e49-44bf-b67e-89cc3243c383 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:01.413748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7ff5e-f128-4572-8824-0796e2d8ac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:01.414548 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7ff5e-f128-4572-8824-0796e2d8ac13 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:01.414832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7ff5e-f128-4572-8824-0796e2d8ac13 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:01.415103 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7ff5e-f128-4572-8824-0796e2d8ac13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:01.416260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7ff5e-f128-4572-8824-0796e2d8ac13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:01.417683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7ff5e-f128-4572-8824-0796e2d8ac13 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:01.419283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 574/1133] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:14:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:01.425214 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b4db5-4971-43c8-9cd7-85e229ce4e4f None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:01.492165 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:01.521326 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b4db5-4971-43c8-9cd7-85e229ce4e4f None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:01.553688 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1d9dd-e3a5-4659-8540-0a9b021fc2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:01.554376 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1d9dd-e3a5-4659-8540-0a9b021fc2fd None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:01.554659 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1d9dd-e3a5-4659-8540-0a9b021fc2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:01.554939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1d9dd-e3a5-4659-8540-0a9b021fc2fd None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:01.555646 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1d9dd-e3a5-4659-8540-0a9b021fc2fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:01.557063 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1d9dd-e3a5-4659-8540-0a9b021fc2fd None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:01.649848 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 560/1134] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Jul 20 15:14:01 2022] POST /identity/v3/projects => generated 311 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:14:03.685061 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44427be6-32c8-48de-b2ec-cb1c7b54bdc1 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:03.685622 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44427be6-32c8-48de-b2ec-cb1c7b54bdc1 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:03.685622 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44427be6-32c8-48de-b2ec-cb1c7b54bdc1 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:03.687011 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 575/1135] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:14:03.692284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b8b73-9067-4594-8a42-5f47378a69d3 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:03.692696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b8b73-9067-4594-8a42-5f47378a69d3 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:03.692844 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b8b73-9067-4594-8a42-5f47378a69d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:03.709082 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-517b8b73-9067-4594-8a42-5f47378a69d3 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:14:03.710293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-517b8b73-9067-4594-8a42-5f47378a69d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:14:03.775554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 561/1136] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:14:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:14:03.801015 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44427be6-32c8-48de-b2ec-cb1c7b54bdc1 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:03.868949 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:03.898897 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44427be6-32c8-48de-b2ec-cb1c7b54bdc1 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:03.932414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04145592-24d4-4c51-9f36-97c7db74fbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:03.933191 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04145592-24d4-4c51-9f36-97c7db74fbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:03.933474 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04145592-24d4-4c51-9f36-97c7db74fbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:03.933751 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04145592-24d4-4c51-9f36-97c7db74fbd8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:03.934919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04145592-24d4-4c51-9f36-97c7db74fbd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:03.936407 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04145592-24d4-4c51-9f36-97c7db74fbd8 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:03.937969 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 576/1137] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:14:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:03.943777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b8b73-9067-4594-8a42-5f47378a69d3 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:03.997093 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:04.041829 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b8b73-9067-4594-8a42-5f47378a69d3 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:04.074166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242d125-c675-4225-9690-686aa29fade8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:04.074856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242d125-c675-4225-9690-686aa29fade8 None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:04.075144 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242d125-c675-4225-9690-686aa29fade8 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:04.075422 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242d125-c675-4225-9690-686aa29fade8 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:04.076216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d242d125-c675-4225-9690-686aa29fade8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:04.077632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d242d125-c675-4225-9690-686aa29fade8 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:04.191336 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 562/1138] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:14:03 2022] POST /identity/v3/users => generated 286 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:14:06.261013 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7f4a7-fcf4-4a68-9a2d-4923609ba7c0 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:06.261552 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7f4a7-fcf4-4a68-9a2d-4923609ba7c0 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:06.261552 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7f4a7-fcf4-4a68-9a2d-4923609ba7c0 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:06.262957 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 577/1139] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:14:06.268736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5aa2c7-2c2a-48e6-9337-421228b90ced None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:06.269161 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5aa2c7-2c2a-48e6-9337-421228b90ced None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:06.269308 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5aa2c7-2c2a-48e6-9337-421228b90ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:06.286171 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-9b5aa2c7-2c2a-48e6-9337-421228b90ced None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:14:06.287414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-9b5aa2c7-2c2a-48e6-9337-421228b90ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:14:06.351799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 563/1140] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:14:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:14:06.375386 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7f4a7-fcf4-4a68-9a2d-4923609ba7c0 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:06.473680 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7f4a7-fcf4-4a68-9a2d-4923609ba7c0 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:06.505369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4773bc-48fc-4dab-b752-f48048cfcbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:06.506137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4773bc-48fc-4dab-b752-f48048cfcbdb None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:06.506442 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4773bc-48fc-4dab-b752-f48048cfcbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:06.506719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4773bc-48fc-4dab-b752-f48048cfcbdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:06.513175 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:06.517495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4773bc-48fc-4dab-b752-f48048cfcbdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:14:06.525005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-fc4773bc-48fc-4dab-b752-f48048cfcbdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:14:06.525005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:14:06.525005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:14:06.525005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:14:06.525005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:14:06.525005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:14:06.525005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:14:06.525005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:14:06.525005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:14:06.525005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:14:06.525005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:14:06.525005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:14:06.525005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:14:06.525005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:14:06.525005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:14:06.525005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:14:06.532744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:14:06.532744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:14:06.532744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:14:06.532744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:14:06.532744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:14:06.532744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:14:06.532744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:14:06.532744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:14:06.532744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:14:06.532744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:14:06.532744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:14:06.532744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:14:06.532744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:14:06.532744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:14:06.532744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:14:06.532744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:14:06.533807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:14:06.533807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:14:06.533807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:14:06.533807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:14:06.533807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:14:06.533807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:14:06.533807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:14:06.533807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:14:06.533807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:14:06.533807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:14:06.533807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:14:06.533807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:14:06.533807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 578/1141] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:14:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:14:06.534743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5aa2c7-2c2a-48e6-9337-421228b90ced None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:06.572373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:06.629094 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5aa2c7-2c2a-48e6-9337-421228b90ced None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:06.662120 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f3022-df7e-4a28-b251-83f9262da7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:06.662806 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f3022-df7e-4a28-b251-83f9262da7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:06.663093 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f3022-df7e-4a28-b251-83f9262da7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:06.663369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f3022-df7e-4a28-b251-83f9262da7b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:06.664115 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f3022-df7e-4a28-b251-83f9262da7b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:14:06.664425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f3022-df7e-4a28-b251-83f9262da7b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:06.665979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f3022-df7e-4a28-b251-83f9262da7b1 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:06.673828 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 564/1142] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:14:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:06.679516 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4773bc-48fc-4dab-b752-f48048cfcbdb None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:06.776666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4773bc-48fc-4dab-b752-f48048cfcbdb None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:06.808227 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5870c7-4011-4bc1-a27e-6e2d13cb5767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:06.809085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5870c7-4011-4bc1-a27e-6e2d13cb5767 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:06.809366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5870c7-4011-4bc1-a27e-6e2d13cb5767 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:06.809639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5870c7-4011-4bc1-a27e-6e2d13cb5767 None admin] PATH_INFO: `/v3/users/bc25ec8d28bd4586ad9341a9e9383b0b` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:06.822659 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5870c7-4011-4bc1-a27e-6e2d13cb5767 None admin] RBAC: Authorizing `identity:get_user(user_id=bc25ec8d28bd4586ad9341a9e9383b0b)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:06.824100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5870c7-4011-4bc1-a27e-6e2d13cb5767 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:06.825699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 579/1143] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:14:06 2022] GET /identity/v3/users/bc25ec8d28bd4586ad9341a9e9383b0b => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:06.832020 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f3022-df7e-4a28-b251-83f9262da7b1 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:06.929606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f3022-df7e-4a28-b251-83f9262da7b1 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:06.931140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:06.963972 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb196b0f-cd60-4147-a671-02d5d3c9894b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:06.964611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb196b0f-cd60-4147-a671-02d5d3c9894b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:06.964899 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb196b0f-cd60-4147-a671-02d5d3c9894b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:06.965176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb196b0f-cd60-4147-a671-02d5d3c9894b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:07.030388 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb196b0f-cd60-4147-a671-02d5d3c9894b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:07.031850 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb196b0f-cd60-4147-a671-02d5d3c9894b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:07.127474 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 565/1144] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:14:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:14:07.133301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5870c7-4011-4bc1-a27e-6e2d13cb5767 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:07.186936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:07.228994 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5870c7-4011-4bc1-a27e-6e2d13cb5767 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:07.264885 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c95a1-9626-43aa-b346-ce4f2283552e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:07.265759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c95a1-9626-43aa-b346-ce4f2283552e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:07.266040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c95a1-9626-43aa-b346-ce4f2283552e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:07.266339 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c95a1-9626-43aa-b346-ce4f2283552e None admin] PATH_INFO: `/v3/projects/e3e19bf7954547ef89295703a79ace3e` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:07.267499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c95a1-9626-43aa-b346-ce4f2283552e None admin] RBAC: Authorizing `identity:get_project(project_id=e3e19bf7954547ef89295703a79ace3e)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:07.268919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c95a1-9626-43aa-b346-ce4f2283552e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:07.270745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 580/1145] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:14:07 2022] GET /identity/v3/projects/e3e19bf7954547ef89295703a79ace3e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:07.276807 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb196b0f-cd60-4147-a671-02d5d3c9894b None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:07.278988 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:07.372339 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb196b0f-cd60-4147-a671-02d5d3c9894b None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:07.404247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6261e6-6528-4dcb-9ca5-8b2d258a5347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:07.404881 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6261e6-6528-4dcb-9ca5-8b2d258a5347 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:07.405162 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6261e6-6528-4dcb-9ca5-8b2d258a5347 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:07.405433 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6261e6-6528-4dcb-9ca5-8b2d258a5347 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:07.406279 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6261e6-6528-4dcb-9ca5-8b2d258a5347 None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=e3e19bf7954547ef89295703a79ace3e, user.id=bc25ec8d28bd4586ad9341a9e9383b0b) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:14:07.407135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6261e6-6528-4dcb-9ca5-8b2d258a5347 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:07.408533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6261e6-6528-4dcb-9ca5-8b2d258a5347 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:07.413704 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 566/1146] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:14:07 2022] GET /identity/v3/role_assignments?user.id=bc25ec8d28bd4586ad9341a9e9383b0b&scope.project.id=e3e19bf7954547ef89295703a79ace3e&role.id=25516be1485c49538544ea8a596e7d17 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:09.396224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6aca5-db22-4234-a964-f5c336cc1bb0 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:09.396721 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6aca5-db22-4234-a964-f5c336cc1bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:09.396721 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6aca5-db22-4234-a964-f5c336cc1bb0 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:09.398036 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 581/1147] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:14:09.403211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a837706-cfa6-4e13-b2a2-c1fdbf9d229f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:09.403589 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a837706-cfa6-4e13-b2a2-c1fdbf9d229f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:09.403763 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a837706-cfa6-4e13-b2a2-c1fdbf9d229f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:09.420052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-3a837706-cfa6-4e13-b2a2-c1fdbf9d229f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:14:09.421218 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-3a837706-cfa6-4e13-b2a2-c1fdbf9d229f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:14:09.477583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:09.485324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 567/1148] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:14:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:14:09.508353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6aca5-db22-4234-a964-f5c336cc1bb0 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:09.608531 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6aca5-db22-4234-a964-f5c336cc1bb0 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:09.631730 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:09.641750 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab545e-4206-46f8-88ee-a8bc0a8ada5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:09.642560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab545e-4206-46f8-88ee-a8bc0a8ada5c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:09.642844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab545e-4206-46f8-88ee-a8bc0a8ada5c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:09.643120 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab545e-4206-46f8-88ee-a8bc0a8ada5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:09.652791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab545e-4206-46f8-88ee-a8bc0a8ada5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:14:09.660831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-28ab545e-4206-46f8-88ee-a8bc0a8ada5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:14:09.660831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:14:09.660831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:14:09.660831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:14:09.660831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:14:09.660831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:14:09.660831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:14:09.660831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:14:09.660831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:14:09.660831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:14:09.660831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:14:09.660831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:14:09.660831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:14:09.660831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:14:09.660831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:14:09.660831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:14:09.661944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:14:09.661944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:14:09.661944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:14:09.661944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:14:09.661944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:14:09.661944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:14:09.661944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:14:09.661944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:14:09.661944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:14:09.661944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:14:09.661944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:14:09.661944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:14:09.661944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:14:09.661944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:14:09.661944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:14:09.661944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:14:09.663205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:14:09.663205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:14:09.663205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:14:09.663205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:14:09.663205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:14:09.663205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:14:09.663205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:14:09.663205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:14:09.663205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:14:09.663205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:14:09.663205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:14:09.663205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:14:09.663205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 582/1149] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:14:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:14:09.668239 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a837706-cfa6-4e13-b2a2-c1fdbf9d229f None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:09.762000 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a837706-cfa6-4e13-b2a2-c1fdbf9d229f None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:09.794530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e20c5-11c4-4f93-b7a7-d675446a645a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:09.795181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e20c5-11c4-4f93-b7a7-d675446a645a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:09.795461 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e20c5-11c4-4f93-b7a7-d675446a645a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:09.795733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e20c5-11c4-4f93-b7a7-d675446a645a None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:09.796469 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e20c5-11c4-4f93-b7a7-d675446a645a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:14:09.796770 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e20c5-11c4-4f93-b7a7-d675446a645a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:09.798697 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e20c5-11c4-4f93-b7a7-d675446a645a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:09.806490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 568/1150] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:14:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:09.812229 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab545e-4206-46f8-88ee-a8bc0a8ada5c None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:09.910689 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab545e-4206-46f8-88ee-a8bc0a8ada5c None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:09.943412 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dcd1b-95c1-46ad-906a-e422f230d669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:09.944275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dcd1b-95c1-46ad-906a-e422f230d669 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:09.944564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dcd1b-95c1-46ad-906a-e422f230d669 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:09.944842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dcd1b-95c1-46ad-906a-e422f230d669 None admin] PATH_INFO: `/v3/users/bc25ec8d28bd4586ad9341a9e9383b0b` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:09.946019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dcd1b-95c1-46ad-906a-e422f230d669 None admin] RBAC: Authorizing `identity:get_user(user_id=bc25ec8d28bd4586ad9341a9e9383b0b)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:09.947490 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dcd1b-95c1-46ad-906a-e422f230d669 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:09.949085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 583/1151] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:14:09 2022] GET /identity/v3/users/bc25ec8d28bd4586ad9341a9e9383b0b => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:09.954916 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e20c5-11c4-4f93-b7a7-d675446a645a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:10.011977 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:10.052681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e20c5-11c4-4f93-b7a7-d675446a645a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:10.085556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1f1da-db54-447d-a956-93794e4d4c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:10.086441 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1f1da-db54-447d-a956-93794e4d4c49 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:10.086723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1f1da-db54-447d-a956-93794e4d4c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:10.087001 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1f1da-db54-447d-a956-93794e4d4c49 None admin] PATH_INFO: `/v3/projects/e3e19bf7954547ef89295703a79ace3e` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:10.088172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1f1da-db54-447d-a956-93794e4d4c49 None admin] RBAC: Authorizing `identity:get_project(project_id=e3e19bf7954547ef89295703a79ace3e)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:10.089631 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1f1da-db54-447d-a956-93794e4d4c49 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:10.092452 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 569/1152] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:14:09 2022] GET /identity/v3/projects/e3e19bf7954547ef89295703a79ace3e => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:10.097731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dcd1b-95c1-46ad-906a-e422f230d669 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:10.148284 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:10.193186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dcd1b-95c1-46ad-906a-e422f230d669 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:10.225798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6a7e8-2ac8-4b90-a81c-3894e407393a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:10.226707 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6a7e8-2ac8-4b90-a81c-3894e407393a None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:10.227059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6a7e8-2ac8-4b90-a81c-3894e407393a None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:10.227272 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6a7e8-2ac8-4b90-a81c-3894e407393a None admin] PATH_INFO: `/v3/projects/e3e19bf7954547ef89295703a79ace3e/users/bc25ec8d28bd4586ad9341a9e9383b0b/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:10.229216 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea6a7e8-2ac8-4b90-a81c-3894e407393a None admin] RBAC: Authorizing `identity:create_grant(project_id=e3e19bf7954547ef89295703a79ace3e, user_id=bc25ec8d28bd4586ad9341a9e9383b0b, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:10.230674 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea6a7e8-2ac8-4b90-a81c-3894e407393a None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:10.338607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 584/1153] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:14:10 2022] PUT /identity/v3/projects/e3e19bf7954547ef89295703a79ace3e/users/bc25ec8d28bd4586ad9341a9e9383b0b/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:14:12.340279 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4916dcf-d0e3-4e2f-a334-ba87984f2c11 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:12.340792 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4916dcf-d0e3-4e2f-a334-ba87984f2c11 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:12.340878 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4916dcf-d0e3-4e2f-a334-ba87984f2c11 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:12.342278 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 570/1154] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:14:12.348047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cbd94-72ea-4874-ab69-f3adb2e91055 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:12.348458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cbd94-72ea-4874-ab69-f3adb2e91055 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:12.348568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cbd94-72ea-4874-ab69-f3adb2e91055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:12.364758 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-600cbd94-72ea-4874-ab69-f3adb2e91055 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:14:12.365920 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-600cbd94-72ea-4874-ab69-f3adb2e91055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:14:12.429371 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 585/1155] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:14:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:14:12.452504 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4916dcf-d0e3-4e2f-a334-ba87984f2c11 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:12.558340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4916dcf-d0e3-4e2f-a334-ba87984f2c11 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:12.583208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:12.591619 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58933a06-4438-4c7e-a637-dca5e3d6d4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:12.592396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58933a06-4438-4c7e-a637-dca5e3d6d4ae None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:12.595632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58933a06-4438-4c7e-a637-dca5e3d6d4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:12.595632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58933a06-4438-4c7e-a637-dca5e3d6d4ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:12.603512 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58933a06-4438-4c7e-a637-dca5e3d6d4ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:14:12.611108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-58933a06-4438-4c7e-a637-dca5e3d6d4ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:14:12.611108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:14:12.611108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:14:12.611108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:14:12.611108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:14:12.611108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:14:12.611108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:14:12.611108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:14:12.611108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:14:12.611108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:14:12.611108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:14:12.611108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:14:12.611108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:14:12.611108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:14:12.611108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:14:12.611108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:14:12.612299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:14:12.612299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:14:12.612299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:14:12.612299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:14:12.612299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:14:12.612299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:14:12.612299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:14:12.612299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:14:12.612299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:14:12.612299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:14:12.612299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:14:12.612299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:14:12.612299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:14:12.612299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:14:12.612299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:14:12.612299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:14:12.613358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:14:12.613358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:14:12.613358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:14:12.613358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:14:12.613358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:14:12.613358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:14:12.613358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:14:12.613358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:14:12.613358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:14:12.613358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:14:12.613358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:14:12.613358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:14:12.613358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 571/1156] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:14:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:14:12.618373 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cbd94-72ea-4874-ab69-f3adb2e91055 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:12.660079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:12.714165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cbd94-72ea-4874-ab69-f3adb2e91055 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:12.746786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62c89f-79a5-4dd6-baea-f6ed1324cc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:12.747446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62c89f-79a5-4dd6-baea-f6ed1324cc6b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:12.747731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62c89f-79a5-4dd6-baea-f6ed1324cc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:12.748006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62c89f-79a5-4dd6-baea-f6ed1324cc6b None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:12.748744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62c89f-79a5-4dd6-baea-f6ed1324cc6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:14:12.749054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62c89f-79a5-4dd6-baea-f6ed1324cc6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:12.751220 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62c89f-79a5-4dd6-baea-f6ed1324cc6b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:12.759096 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 586/1157] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:14:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:12.764851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58933a06-4438-4c7e-a637-dca5e3d6d4ae None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:12.868044 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58933a06-4438-4c7e-a637-dca5e3d6d4ae None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:12.903500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b6c17-757e-44f2-a6e1-a2c917c6df82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:12.904356 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b6c17-757e-44f2-a6e1-a2c917c6df82 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:12.904642 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b6c17-757e-44f2-a6e1-a2c917c6df82 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:12.904920 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b6c17-757e-44f2-a6e1-a2c917c6df82 None admin] PATH_INFO: `/v3/users/bc25ec8d28bd4586ad9341a9e9383b0b` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:12.906135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b6c17-757e-44f2-a6e1-a2c917c6df82 None admin] RBAC: Authorizing `identity:get_user(user_id=bc25ec8d28bd4586ad9341a9e9383b0b)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:12.907615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b6c17-757e-44f2-a6e1-a2c917c6df82 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:12.909276 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 572/1158] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:14:12 2022] GET /identity/v3/users/bc25ec8d28bd4586ad9341a9e9383b0b => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:12.915214 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62c89f-79a5-4dd6-baea-f6ed1324cc6b None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:13.011844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62c89f-79a5-4dd6-baea-f6ed1324cc6b None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:13.014119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:13.044298 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaed393-9636-4ae9-ab93-7b60686090ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:13.044927 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaed393-9636-4ae9-ab93-7b60686090ad None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:13.045205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaed393-9636-4ae9-ab93-7b60686090ad None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:13.045475 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaed393-9636-4ae9-ab93-7b60686090ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:13.109073 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaed393-9636-4ae9-ab93-7b60686090ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:13.111166 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaed393-9636-4ae9-ab93-7b60686090ad None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:13.206999 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 587/1159] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:14:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:14:13.212837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b6c17-757e-44f2-a6e1-a2c917c6df82 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:13.266083 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:13.308407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b6c17-757e-44f2-a6e1-a2c917c6df82 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:13.340156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c49a0-3826-4639-a2ed-578cb46c3f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:13.340995 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c49a0-3826-4639-a2ed-578cb46c3f62 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:13.341533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c49a0-3826-4639-a2ed-578cb46c3f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:13.341533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c49a0-3826-4639-a2ed-578cb46c3f62 None admin] PATH_INFO: `/v3/projects/e3e19bf7954547ef89295703a79ace3e` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:13.342714 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c49a0-3826-4639-a2ed-578cb46c3f62 None admin] RBAC: Authorizing `identity:get_project(project_id=e3e19bf7954547ef89295703a79ace3e)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:13.344181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c49a0-3826-4639-a2ed-578cb46c3f62 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:13.345828 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 573/1160] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:14:13 2022] GET /identity/v3/projects/e3e19bf7954547ef89295703a79ace3e => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:13.351879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaed393-9636-4ae9-ab93-7b60686090ad None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:13.354920 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:13.448616 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaed393-9636-4ae9-ab93-7b60686090ad None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:13.480842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b937ee9-d6b0-48a5-bc62-de78b099d07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:13.481431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b937ee9-d6b0-48a5-bc62-de78b099d07e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:13.481718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b937ee9-d6b0-48a5-bc62-de78b099d07e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:13.481996 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b937ee9-d6b0-48a5-bc62-de78b099d07e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:13.482846 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b937ee9-d6b0-48a5-bc62-de78b099d07e None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=e3e19bf7954547ef89295703a79ace3e, user.id=bc25ec8d28bd4586ad9341a9e9383b0b) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:14:13.483163 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b937ee9-d6b0-48a5-bc62-de78b099d07e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:13.485208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b937ee9-d6b0-48a5-bc62-de78b099d07e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:13.490881 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 588/1161] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:14:13 2022] GET /identity/v3/role_assignments?user.id=bc25ec8d28bd4586ad9341a9e9383b0b&scope.project.id=e3e19bf7954547ef89295703a79ace3e&role.id=25516be1485c49538544ea8a596e7d17 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:15.509311 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a43b01-fedb-4405-ab73-7ddfded47a8a None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:15.509811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a43b01-fedb-4405-ab73-7ddfded47a8a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:15.509811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a43b01-fedb-4405-ab73-7ddfded47a8a None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:15.511296 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 574/1162] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:14:15.516488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905d2ef-789f-4690-8374-d08497beba73 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:15.516909 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905d2ef-789f-4690-8374-d08497beba73 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:15.517037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905d2ef-789f-4690-8374-d08497beba73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:15.533926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-a905d2ef-789f-4690-8374-d08497beba73 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:14:15.535169 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-a905d2ef-789f-4690-8374-d08497beba73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:14:15.593920 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:15.598880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 589/1163] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:14:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:14:15.621489 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a43b01-fedb-4405-ab73-7ddfded47a8a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:15.721554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a43b01-fedb-4405-ab73-7ddfded47a8a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:15.744212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:15.755422 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503d43d-ef77-4fd6-8293-e6f3a761e264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:15.756057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503d43d-ef77-4fd6-8293-e6f3a761e264 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:15.756346 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503d43d-ef77-4fd6-8293-e6f3a761e264 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:15.756625 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503d43d-ef77-4fd6-8293-e6f3a761e264 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:15.821385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503d43d-ef77-4fd6-8293-e6f3a761e264 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:15.822862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503d43d-ef77-4fd6-8293-e6f3a761e264 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:15.919015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 575/1164] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:14:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:14:15.925555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905d2ef-789f-4690-8374-d08497beba73 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:16.019707 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905d2ef-789f-4690-8374-d08497beba73 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:16.051816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852b056-e1c0-4b8f-8478-7e11d906b59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:16.052594 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852b056-e1c0-4b8f-8478-7e11d906b59c None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:16.052873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852b056-e1c0-4b8f-8478-7e11d906b59c None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:16.053141 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852b056-e1c0-4b8f-8478-7e11d906b59c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:16.059556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852b056-e1c0-4b8f-8478-7e11d906b59c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:16.060974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852b056-e1c0-4b8f-8478-7e11d906b59c None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:16.074704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-c852b056-e1c0-4b8f-8478-7e11d906b59c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 15:14:16.074704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:14:16.074704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 20 15:14:16.074704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 20 15:14:16.074704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:14:16.074704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:14:16.074704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:14:16.074704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:14:16.074704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 20 15:14:16.074704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:14:16.074704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:14:16.074704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:14:16.074704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:14:16.074704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:14:16.074704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:14:16.074704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:14:16.074704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:14:16.075901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:14:16.075901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:14:16.075901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:14:16.075901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:14:16.075901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:14:16.075901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:14:16.075901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:14:16.075901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:14:16.075901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:14:16.075901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:14:16.075901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:14:16.075901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:14:16.075901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:14:16.075901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:14:16.075901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:14:16.075901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:14:16.076828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:14:16.076828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:14:16.076828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:14:16.076828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:14:16.076828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:14:16.076828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:14:16.076828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:14:16.076828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:14:16.076828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:14:16.076828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:14:16.076828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 20 15:14:16.076828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:14:16.076828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 15:14:16.076828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:14:16.076828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 590/1165] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Jul 20 15:14:15 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:14:16.082497 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503d43d-ef77-4fd6-8293-e6f3a761e264 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:16.106165 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:16.179952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503d43d-ef77-4fd6-8293-e6f3a761e264 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:16.212905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1980f1d-6517-4b46-b836-eb4fcc2166c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:16.213560 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1980f1d-6517-4b46-b836-eb4fcc2166c9 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:16.213844 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1980f1d-6517-4b46-b836-eb4fcc2166c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:16.214122 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1980f1d-6517-4b46-b836-eb4fcc2166c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:16.214872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1980f1d-6517-4b46-b836-eb4fcc2166c9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:14:16.215184 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1980f1d-6517-4b46-b836-eb4fcc2166c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:16.216561 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1980f1d-6517-4b46-b836-eb4fcc2166c9 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:16.224144 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 576/1166] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Jul 20 15:14:16 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:18.224807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6ac38-48e0-4287-8862-8dea14e842a7 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:18.225381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6ac38-48e0-4287-8862-8dea14e842a7 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:18.225381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6ac38-48e0-4287-8862-8dea14e842a7 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:18.227092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 591/1167] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:14:18.232410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033d52e-4e92-4edf-aeae-29e2b06fb933 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:18.232822 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033d52e-4e92-4edf-aeae-29e2b06fb933 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:18.232976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033d52e-4e92-4edf-aeae-29e2b06fb933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:18.250058 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-f033d52e-4e92-4edf-aeae-29e2b06fb933 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:14:18.251270 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f033d52e-4e92-4edf-aeae-29e2b06fb933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:14:18.315821 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 577/1168] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:14:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:14:18.338864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6ac38-48e0-4287-8862-8dea14e842a7 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:18.396203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:18.436486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6ac38-48e0-4287-8862-8dea14e842a7 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:18.469044 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b7455-8bab-4557-b827-b593a9c38cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:18.469689 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b7455-8bab-4557-b827-b593a9c38cab None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:18.469971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b7455-8bab-4557-b827-b593a9c38cab None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:18.470243 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b7455-8bab-4557-b827-b593a9c38cab None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:18.471001 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b7455-8bab-4557-b827-b593a9c38cab None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:18.472358 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b7455-8bab-4557-b827-b593a9c38cab None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:18.562982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 592/1169] 10.222.0.78 () {60 vars in 1138 bytes} [Wed Jul 20 15:14:18 2022] POST /identity/v3/domains => generated 267 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:14:20.589829 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32488e42-0189-43b8-806c-23c62065cc41 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:20.590396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32488e42-0189-43b8-806c-23c62065cc41 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:20.590503 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32488e42-0189-43b8-806c-23c62065cc41 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:20.591798 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 578/1170] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:14:20.597046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b483d-ca28-4977-a333-d503cfdd37fc None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:20.597459 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b483d-ca28-4977-a333-d503cfdd37fc None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:20.597604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b483d-ca28-4977-a333-d503cfdd37fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:20.614981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-e20b483d-ca28-4977-a333-d503cfdd37fc None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:14:20.616169 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e20b483d-ca28-4977-a333-d503cfdd37fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:14:20.680495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 593/1171] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:14:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:14:20.704364 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32488e42-0189-43b8-806c-23c62065cc41 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:20.721866 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:20.802802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32488e42-0189-43b8-806c-23c62065cc41 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:20.835008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee24fa-b0d9-4127-8063-6aa2844aa04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:20.835860 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee24fa-b0d9-4127-8063-6aa2844aa04b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:20.836147 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee24fa-b0d9-4127-8063-6aa2844aa04b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:20.836439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee24fa-b0d9-4127-8063-6aa2844aa04b None admin] PATH_INFO: `/v3/domains/d7a7b8c32d1e46d380bfd2ff42987d4b` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:20.846807 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee24fa-b0d9-4127-8063-6aa2844aa04b None admin] RBAC: Authorizing `identity:get_domain(domain_id=d7a7b8c32d1e46d380bfd2ff42987d4b)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:20.848203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee24fa-b0d9-4127-8063-6aa2844aa04b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:20.849828 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 579/1172] 10.222.0.78 () {56 vars in 1183 bytes} [Wed Jul 20 15:14:20 2022] GET /identity/v3/domains/d7a7b8c32d1e46d380bfd2ff42987d4b => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:20.855692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b483d-ca28-4977-a333-d503cfdd37fc None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:20.903582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:20.953288 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b483d-ca28-4977-a333-d503cfdd37fc None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:20.987516 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174f0c5-fb35-49e3-9687-fae4170e0533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:20.988177 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174f0c5-fb35-49e3-9687-fae4170e0533 None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:20.988464 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174f0c5-fb35-49e3-9687-fae4170e0533 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:20.988745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174f0c5-fb35-49e3-9687-fae4170e0533 None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:20.989452 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174f0c5-fb35-49e3-9687-fae4170e0533 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:20.990948 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174f0c5-fb35-49e3-9687-fae4170e0533 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:21.099785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 594/1173] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Jul 20 15:14:20 2022] POST /identity/v3/projects => generated 361 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:14:23.155824 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8470d-8e38-424d-a968-30e922a50b46 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:23.156369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8470d-8e38-424d-a968-30e922a50b46 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:23.156475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8470d-8e38-424d-a968-30e922a50b46 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:23.157765 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 580/1174] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:14:23.163009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abee7a-915a-4575-acda-05e094835a18 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:23.163404 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abee7a-915a-4575-acda-05e094835a18 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:23.163589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abee7a-915a-4575-acda-05e094835a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:23.180390 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-92abee7a-915a-4575-acda-05e094835a18 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:14:23.181581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-92abee7a-915a-4575-acda-05e094835a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:14:23.246253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 595/1175] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:14:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:14:23.271636 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8470d-8e38-424d-a968-30e922a50b46 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:23.346053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:23.368240 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8470d-8e38-424d-a968-30e922a50b46 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:23.401750 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f0731-c8c8-4dd0-93c5-cc2f0ae7c69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:23.402626 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f0731-c8c8-4dd0-93c5-cc2f0ae7c69a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:23.402913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f0731-c8c8-4dd0-93c5-cc2f0ae7c69a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:23.403189 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f0731-c8c8-4dd0-93c5-cc2f0ae7c69a None admin] PATH_INFO: `/v3/domains/d7a7b8c32d1e46d380bfd2ff42987d4b` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:23.404357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f0731-c8c8-4dd0-93c5-cc2f0ae7c69a None admin] RBAC: Authorizing `identity:get_domain(domain_id=d7a7b8c32d1e46d380bfd2ff42987d4b)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:23.405806 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f0731-c8c8-4dd0-93c5-cc2f0ae7c69a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:23.407427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 581/1176] 10.222.0.78 () {56 vars in 1183 bytes} [Wed Jul 20 15:14:23 2022] GET /identity/v3/domains/d7a7b8c32d1e46d380bfd2ff42987d4b => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:23.413181 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abee7a-915a-4575-acda-05e094835a18 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:23.441885 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:23.509155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abee7a-915a-4575-acda-05e094835a18 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:23.541254 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb20ce9-b4d6-4d65-8fd1-bc0f42fb9ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:23.541906 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb20ce9-b4d6-4d65-8fd1-bc0f42fb9ea0 None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:23.542188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb20ce9-b4d6-4d65-8fd1-bc0f42fb9ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:23.542495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb20ce9-b4d6-4d65-8fd1-bc0f42fb9ea0 None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:23.543210 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb20ce9-b4d6-4d65-8fd1-bc0f42fb9ea0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:23.544631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb20ce9-b4d6-4d65-8fd1-bc0f42fb9ea0 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:23.641500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 596/1177] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:14:23 2022] POST /identity/v3/users => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:14:25.714429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64809f4-8797-4e8b-8dc4-3261b2a88f03 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:25.714950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64809f4-8797-4e8b-8dc4-3261b2a88f03 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:25.715062 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64809f4-8797-4e8b-8dc4-3261b2a88f03 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:25.716358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 582/1178] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:14:25.721490 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc14f4-fdac-40d0-985a-6d77cd1188d2 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:25.721925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc14f4-fdac-40d0-985a-6d77cd1188d2 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:25.722052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc14f4-fdac-40d0-985a-6d77cd1188d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:25.738826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-d3fc14f4-fdac-40d0-985a-6d77cd1188d2 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:14:25.740005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d3fc14f4-fdac-40d0-985a-6d77cd1188d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:14:25.804595 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 597/1179] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:14:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:14:25.829515 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64809f4-8797-4e8b-8dc4-3261b2a88f03 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:25.925772 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64809f4-8797-4e8b-8dc4-3261b2a88f03 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:25.957742 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a04d73-b816-4169-bbdd-68252a981353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:25.958534 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a04d73-b816-4169-bbdd-68252a981353 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:25.958815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a04d73-b816-4169-bbdd-68252a981353 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:25.959087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a04d73-b816-4169-bbdd-68252a981353 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:25.968878 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a04d73-b816-4169-bbdd-68252a981353 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:14:25.976414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-80a04d73-b816-4169-bbdd-68252a981353 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:14:25.976414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:14:25.976414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:14:25.976414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:14:25.976414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:14:25.976414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:14:25.976414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:14:25.976414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:14:25.976414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:14:25.976414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:14:25.976414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:14:25.976414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:14:25.976414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:14:25.976414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:14:25.976414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:14:25.976414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:14:25.977644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:14:25.977644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:14:25.977644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:14:25.977644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:14:25.977644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:14:25.977644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:14:25.977644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:14:25.977644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:14:25.977644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:14:25.977644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:14:25.977644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:14:25.977644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:14:25.977644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:14:25.977644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:14:25.977644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:14:25.977644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:14:25.978645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:14:25.978645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:14:25.978645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:14:25.978645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:14:25.978645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:14:25.978645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:14:25.978645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:14:25.978645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:14:25.978645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:14:25.978645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:14:25.978645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:14:25.978645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:14:25.978645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 583/1180] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:14:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:14:25.983385 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc14f4-fdac-40d0-985a-6d77cd1188d2 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:26.001623 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:26.079935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc14f4-fdac-40d0-985a-6d77cd1188d2 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:26.114329 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e95fb-dcb8-435c-a046-0ee31b222065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:26.114995 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e95fb-dcb8-435c-a046-0ee31b222065 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:26.115276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e95fb-dcb8-435c-a046-0ee31b222065 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:26.115551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e95fb-dcb8-435c-a046-0ee31b222065 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:26.116509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e95fb-dcb8-435c-a046-0ee31b222065 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:14:26.116851 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e95fb-dcb8-435c-a046-0ee31b222065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:26.118499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e95fb-dcb8-435c-a046-0ee31b222065 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:26.127182 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 598/1181] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:14:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:26.133055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a04d73-b816-4169-bbdd-68252a981353 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:26.136798 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:26.228665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a04d73-b816-4169-bbdd-68252a981353 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:26.260951 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9993cca-f9d8-49fa-8c82-72b976dcee4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:26.261811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9993cca-f9d8-49fa-8c82-72b976dcee4a None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:26.262093 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9993cca-f9d8-49fa-8c82-72b976dcee4a None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:26.262399 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9993cca-f9d8-49fa-8c82-72b976dcee4a None admin] PATH_INFO: `/v3/users/f048406d918e4d4a9f45a0afd6a7e9de` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:26.274948 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9993cca-f9d8-49fa-8c82-72b976dcee4a None admin] RBAC: Authorizing `identity:get_user(user_id=f048406d918e4d4a9f45a0afd6a7e9de)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:26.276377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9993cca-f9d8-49fa-8c82-72b976dcee4a None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:26.277994 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 584/1182] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:14:26 2022] GET /identity/v3/users/f048406d918e4d4a9f45a0afd6a7e9de => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:26.284219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e95fb-dcb8-435c-a046-0ee31b222065 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:26.361569 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:26.382705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e95fb-dcb8-435c-a046-0ee31b222065 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:26.416069 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867e2df-b1f7-4a0f-8c19-9466a3000d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:26.416777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867e2df-b1f7-4a0f-8c19-9466a3000d53 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:26.417063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867e2df-b1f7-4a0f-8c19-9466a3000d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:26.417342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867e2df-b1f7-4a0f-8c19-9466a3000d53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:26.483820 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867e2df-b1f7-4a0f-8c19-9466a3000d53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:26.485736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867e2df-b1f7-4a0f-8c19-9466a3000d53 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:26.558931 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:26.580371 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 599/1183] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:14:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:14:26.586306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9993cca-f9d8-49fa-8c82-72b976dcee4a None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:26.646808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:26.686177 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9993cca-f9d8-49fa-8c82-72b976dcee4a None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:26.719549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1005ff-00ea-4b6c-96c2-ad0e27886fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:26.720436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1005ff-00ea-4b6c-96c2-ad0e27886fac None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:26.720722 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1005ff-00ea-4b6c-96c2-ad0e27886fac None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:26.721002 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1005ff-00ea-4b6c-96c2-ad0e27886fac None admin] PATH_INFO: `/v3/projects/5974411fdb2b469ab901d8e72eb1844e` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:26.722224 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1005ff-00ea-4b6c-96c2-ad0e27886fac None admin] RBAC: Authorizing `identity:get_project(project_id=5974411fdb2b469ab901d8e72eb1844e)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:26.723717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1005ff-00ea-4b6c-96c2-ad0e27886fac None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:26.725394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 585/1184] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:14:26 2022] GET /identity/v3/projects/5974411fdb2b469ab901d8e72eb1844e => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:26.731392 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867e2df-b1f7-4a0f-8c19-9466a3000d53 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:26.832674 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867e2df-b1f7-4a0f-8c19-9466a3000d53 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:26.865275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41668c-900f-478c-aecd-4c264dc80637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:26.865928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41668c-900f-478c-aecd-4c264dc80637 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:26.866213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41668c-900f-478c-aecd-4c264dc80637 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:26.866525 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41668c-900f-478c-aecd-4c264dc80637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:26.867347 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f41668c-900f-478c-aecd-4c264dc80637 None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=5974411fdb2b469ab901d8e72eb1844e, user.id=f048406d918e4d4a9f45a0afd6a7e9de) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:14:26.867662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f41668c-900f-478c-aecd-4c264dc80637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:26.869123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f41668c-900f-478c-aecd-4c264dc80637 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:26.874935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 600/1185] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:14:26 2022] GET /identity/v3/role_assignments?user.id=f048406d918e4d4a9f45a0afd6a7e9de&scope.project.id=5974411fdb2b469ab901d8e72eb1844e&role.id=25516be1485c49538544ea8a596e7d17 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:28.868102 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c5b0b-e11b-4218-8d8d-cb25573293a3 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:28.868661 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c5b0b-e11b-4218-8d8d-cb25573293a3 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:28.868661 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c5b0b-e11b-4218-8d8d-cb25573293a3 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:28.870062 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 586/1186] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:14:28.875988 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e917c-f81c-4d14-b807-5a33c7c46af7 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:28.876377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e917c-f81c-4d14-b807-5a33c7c46af7 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:28.876569 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e917c-f81c-4d14-b807-5a33c7c46af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:28.892982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-9a3e917c-f81c-4d14-b807-5a33c7c46af7 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:14:28.894164 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9a3e917c-f81c-4d14-b807-5a33c7c46af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:14:28.928163 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:28.958699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 601/1187] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:14:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:14:28.980981 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c5b0b-e11b-4218-8d8d-cb25573293a3 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:29.081695 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c5b0b-e11b-4218-8d8d-cb25573293a3 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:29.109876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:29.115439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b312b5c-2940-42a4-96c5-bcf857ba38ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:29.116226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b312b5c-2940-42a4-96c5-bcf857ba38ec None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:29.122459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b312b5c-2940-42a4-96c5-bcf857ba38ec None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:29.122459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b312b5c-2940-42a4-96c5-bcf857ba38ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:29.127497 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b312b5c-2940-42a4-96c5-bcf857ba38ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:14:29.135436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-0b312b5c-2940-42a4-96c5-bcf857ba38ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:14:29.135436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:14:29.135436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:14:29.135436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:14:29.135436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:14:29.135436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:14:29.135436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:14:29.135436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:14:29.135436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:14:29.135436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:14:29.135436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:14:29.135436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:14:29.135436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:14:29.135436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:14:29.135436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:14:29.135436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:14:29.136396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:14:29.136396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:14:29.136396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:14:29.136396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:14:29.136396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:14:29.136396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:14:29.136396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:14:29.136396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:14:29.136396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:14:29.136396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:14:29.136396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:14:29.136396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:14:29.136396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:14:29.136396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:14:29.136396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:14:29.136396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:14:29.137340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:14:29.137340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:14:29.137340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:14:29.137340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:14:29.137340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:14:29.137340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:14:29.137340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:14:29.137340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:14:29.137340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:14:29.137340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:14:29.137340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:14:29.137340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:14:29.137340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 587/1188] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:14:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:14:29.142866 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e917c-f81c-4d14-b807-5a33c7c46af7 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:29.239258 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e917c-f81c-4d14-b807-5a33c7c46af7 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:29.272055 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6616f28-897c-4e3c-aabe-a45482e00414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:29.272713 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6616f28-897c-4e3c-aabe-a45482e00414 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:29.272996 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6616f28-897c-4e3c-aabe-a45482e00414 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:29.273265 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6616f28-897c-4e3c-aabe-a45482e00414 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:29.273995 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6616f28-897c-4e3c-aabe-a45482e00414 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:14:29.274322 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6616f28-897c-4e3c-aabe-a45482e00414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:29.275680 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6616f28-897c-4e3c-aabe-a45482e00414 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:29.283883 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 602/1189] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:14:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:29.289591 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b312b5c-2940-42a4-96c5-bcf857ba38ec None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:29.386725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b312b5c-2940-42a4-96c5-bcf857ba38ec None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:29.419448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046be33-0e40-4f9c-8254-b9bdae311501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:29.420299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046be33-0e40-4f9c-8254-b9bdae311501 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:29.420585 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046be33-0e40-4f9c-8254-b9bdae311501 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:29.420864 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046be33-0e40-4f9c-8254-b9bdae311501 None admin] PATH_INFO: `/v3/users/f048406d918e4d4a9f45a0afd6a7e9de` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:29.422670 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046be33-0e40-4f9c-8254-b9bdae311501 None admin] RBAC: Authorizing `identity:get_user(user_id=f048406d918e4d4a9f45a0afd6a7e9de)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:29.424111 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046be33-0e40-4f9c-8254-b9bdae311501 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:29.425722 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 588/1190] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:14:29 2022] GET /identity/v3/users/f048406d918e4d4a9f45a0afd6a7e9de => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:29.431732 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6616f28-897c-4e3c-aabe-a45482e00414 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:29.464804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:29.527394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6616f28-897c-4e3c-aabe-a45482e00414 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:29.559405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c72768-2d7d-499e-911c-6388a634c8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:29.560258 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c72768-2d7d-499e-911c-6388a634c8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:29.560527 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c72768-2d7d-499e-911c-6388a634c8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:29.560798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c72768-2d7d-499e-911c-6388a634c8e3 None admin] PATH_INFO: `/v3/projects/5974411fdb2b469ab901d8e72eb1844e` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:29.561978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c72768-2d7d-499e-911c-6388a634c8e3 None admin] RBAC: Authorizing `identity:get_project(project_id=5974411fdb2b469ab901d8e72eb1844e)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:29.563423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c72768-2d7d-499e-911c-6388a634c8e3 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:29.565057 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 603/1191] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:14:29 2022] GET /identity/v3/projects/5974411fdb2b469ab901d8e72eb1844e => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:29.570910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046be33-0e40-4f9c-8254-b9bdae311501 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:29.627533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:29.667932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046be33-0e40-4f9c-8254-b9bdae311501 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:29.699282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e6905-d12c-4092-82a8-ecd9439f4071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:29.700153 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e6905-d12c-4092-82a8-ecd9439f4071 None admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:29.700435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e6905-d12c-4092-82a8-ecd9439f4071 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:29.700712 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e6905-d12c-4092-82a8-ecd9439f4071 None admin] PATH_INFO: `/v3/projects/5974411fdb2b469ab901d8e72eb1844e/users/f048406d918e4d4a9f45a0afd6a7e9de/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:29.703197 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e6905-d12c-4092-82a8-ecd9439f4071 None admin] RBAC: Authorizing `identity:create_grant(project_id=5974411fdb2b469ab901d8e72eb1844e, user_id=f048406d918e4d4a9f45a0afd6a7e9de, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:29.704603 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e6905-d12c-4092-82a8-ecd9439f4071 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:29.800418 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 589/1192] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:14:29 2022] PUT /identity/v3/projects/5974411fdb2b469ab901d8e72eb1844e/users/f048406d918e4d4a9f45a0afd6a7e9de/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:14:31.791556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a4c1d-df08-46fd-b6e3-5f0117c5e1ec None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:31.792109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a4c1d-df08-46fd-b6e3-5f0117c5e1ec None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:31.792109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a4c1d-df08-46fd-b6e3-5f0117c5e1ec None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:31.793445 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 604/1193] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:14:31.799316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dec0ed-13a3-4920-91a5-05d0fef917e5 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:31.799712 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dec0ed-13a3-4920-91a5-05d0fef917e5 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:31.799884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dec0ed-13a3-4920-91a5-05d0fef917e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:31.817698 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-17dec0ed-13a3-4920-91a5-05d0fef917e5 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:14:31.818641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-17dec0ed-13a3-4920-91a5-05d0fef917e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:14:31.883724 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 590/1194] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:14:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:14:31.907173 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a4c1d-df08-46fd-b6e3-5f0117c5e1ec None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:32.004685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a4c1d-df08-46fd-b6e3-5f0117c5e1ec None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:32.006149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:32.036531 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4241bf-b84e-4ff4-8f35-02ab090c2d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:32.037303 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4241bf-b84e-4ff4-8f35-02ab090c2d63 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:32.037586 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4241bf-b84e-4ff4-8f35-02ab090c2d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:32.037861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4241bf-b84e-4ff4-8f35-02ab090c2d63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:32.048593 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4241bf-b84e-4ff4-8f35-02ab090c2d63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:14:32.056104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-0c4241bf-b84e-4ff4-8f35-02ab090c2d63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:14:32.056104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:14:32.056104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:14:32.056104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:14:32.056104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:14:32.056104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:14:32.056104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:14:32.056104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:14:32.056104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:14:32.056104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:14:32.056104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:14:32.056104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:14:32.056104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:14:32.056104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:14:32.056104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:14:32.056104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:14:32.057213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:14:32.057213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:14:32.057213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:14:32.057213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:14:32.057213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:14:32.057213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:14:32.057213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:14:32.057213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:14:32.057213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:14:32.057213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:14:32.057213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:14:32.057213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:14:32.057213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:14:32.057213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:14:32.057213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:14:32.057213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:14:32.058289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:14:32.058289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:14:32.058289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:14:32.058289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:14:32.058289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:14:32.058289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:14:32.058289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:14:32.058289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:14:32.058289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:14:32.058289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:14:32.058289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:14:32.058289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:14:32.058289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 605/1195] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:14:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:14:32.063656 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dec0ed-13a3-4920-91a5-05d0fef917e5 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:32.111872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:32.173214 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dec0ed-13a3-4920-91a5-05d0fef917e5 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:32.205381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7475f8-c4e2-4579-9e78-f11ad4c6fc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:32.206036 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7475f8-c4e2-4579-9e78-f11ad4c6fc7e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:32.206345 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7475f8-c4e2-4579-9e78-f11ad4c6fc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:32.206623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7475f8-c4e2-4579-9e78-f11ad4c6fc7e None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:32.207357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7475f8-c4e2-4579-9e78-f11ad4c6fc7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:14:32.207660 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7475f8-c4e2-4579-9e78-f11ad4c6fc7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:32.209016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7475f8-c4e2-4579-9e78-f11ad4c6fc7e None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:32.218316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 591/1196] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:14:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:32.224341 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4241bf-b84e-4ff4-8f35-02ab090c2d63 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:32.319929 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4241bf-b84e-4ff4-8f35-02ab090c2d63 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:32.352198 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d1fe3-f33b-4a14-b9e2-558c0169b067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:32.353050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d1fe3-f33b-4a14-b9e2-558c0169b067 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:32.353333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d1fe3-f33b-4a14-b9e2-558c0169b067 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:32.353610 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d1fe3-f33b-4a14-b9e2-558c0169b067 None admin] PATH_INFO: `/v3/users/f048406d918e4d4a9f45a0afd6a7e9de` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:32.354854 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d1fe3-f33b-4a14-b9e2-558c0169b067 None admin] RBAC: Authorizing `identity:get_user(user_id=f048406d918e4d4a9f45a0afd6a7e9de)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:32.356286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d1fe3-f33b-4a14-b9e2-558c0169b067 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:32.357864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 606/1197] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Jul 20 15:14:32 2022] GET /identity/v3/users/f048406d918e4d4a9f45a0afd6a7e9de => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:32.364148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7475f8-c4e2-4579-9e78-f11ad4c6fc7e None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:32.460918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7475f8-c4e2-4579-9e78-f11ad4c6fc7e None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:32.471738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:32.495032 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47bf30-c3c8-49f3-a610-43afa89b4325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:32.495670 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47bf30-c3c8-49f3-a610-43afa89b4325 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:32.495954 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47bf30-c3c8-49f3-a610-43afa89b4325 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:32.496229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47bf30-c3c8-49f3-a610-43afa89b4325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:32.560931 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47bf30-c3c8-49f3-a610-43afa89b4325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:32.562420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47bf30-c3c8-49f3-a610-43afa89b4325 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:32.656572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 592/1198] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:14:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:14:32.663495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d1fe3-f33b-4a14-b9e2-558c0169b067 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:32.693052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:32.760042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d1fe3-f33b-4a14-b9e2-558c0169b067 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:32.793233 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6aba8-3f68-4885-b5ca-c19bbd1d5fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:32.794082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6aba8-3f68-4885-b5ca-c19bbd1d5fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:32.794396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6aba8-3f68-4885-b5ca-c19bbd1d5fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:32.794676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6aba8-3f68-4885-b5ca-c19bbd1d5fb6 None admin] PATH_INFO: `/v3/projects/5974411fdb2b469ab901d8e72eb1844e` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:32.795868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6aba8-3f68-4885-b5ca-c19bbd1d5fb6 None admin] RBAC: Authorizing `identity:get_project(project_id=5974411fdb2b469ab901d8e72eb1844e)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:32.797358 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6aba8-3f68-4885-b5ca-c19bbd1d5fb6 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:32.799059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 607/1199] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Jul 20 15:14:32 2022] GET /identity/v3/projects/5974411fdb2b469ab901d8e72eb1844e => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:32.805000 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47bf30-c3c8-49f3-a610-43afa89b4325 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:14:32.816227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:32.902867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47bf30-c3c8-49f3-a610-43afa89b4325 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:14:32.934592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba171f-1a43-466e-9b47-e6454d9fbc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:14:32.935166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba171f-1a43-466e-9b47-e6454d9fbc60 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:14:32.935444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba171f-1a43-466e-9b47-e6454d9fbc60 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:14:32.935720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba171f-1a43-466e-9b47-e6454d9fbc60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:14:32.937042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba171f-1a43-466e-9b47-e6454d9fbc60 None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=5974411fdb2b469ab901d8e72eb1844e, user.id=f048406d918e4d4a9f45a0afd6a7e9de) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:14:32.937350 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba171f-1a43-466e-9b47-e6454d9fbc60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:14:32.938778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba171f-1a43-466e-9b47-e6454d9fbc60 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:14:32.944333 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 593/1200] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:14:32 2022] GET /identity/v3/role_assignments?user.id=f048406d918e4d4a9f45a0afd6a7e9de&scope.project.id=5974411fdb2b469ab901d8e72eb1844e&role.id=25516be1485c49538544ea8a596e7d17 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:14:42.789334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=51 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:42.941761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=63 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:45.297255 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf46aa4-09f1-48c3-841e-1ec893e3be55 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:45.297846 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf46aa4-09f1-48c3-841e-1ec893e3be55 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:45.297846 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf46aa4-09f1-48c3-841e-1ec893e3be55 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:45.299180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 608/1201] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:15:45.304348 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ca0d7-f807-4a10-9965-82937e500ebe None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:45.305372 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ca0d7-f807-4a10-9965-82937e500ebe None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:45.305372 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ca0d7-f807-4a10-9965-82937e500ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:45.321216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-db0ca0d7-f807-4a10-9965-82937e500ebe None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:15:45.322450 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-db0ca0d7-f807-4a10-9965-82937e500ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:15:45.389237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 594/1202] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:15:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:15:45.414250 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf46aa4-09f1-48c3-841e-1ec893e3be55 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:45.514987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf46aa4-09f1-48c3-841e-1ec893e3be55 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:45.548261 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81bfd1-7119-46d6-88e9-5938ac884abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:45.549048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81bfd1-7119-46d6-88e9-5938ac884abb None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:45.549358 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81bfd1-7119-46d6-88e9-5938ac884abb None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:45.549638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81bfd1-7119-46d6-88e9-5938ac884abb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:45.559713 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81bfd1-7119-46d6-88e9-5938ac884abb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:45.561159 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81bfd1-7119-46d6-88e9-5938ac884abb None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:45.580118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-ba81bfd1-7119-46d6-88e9-5938ac884abb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:15:45.580118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:15:45.580118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:15:45.580118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:15:45.580118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:15:45.580118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:15:45.580118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:15:45.580118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:15:45.580118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:15:45.580118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:15:45.580118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:15:45.580118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:15:45.580118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:15:45.580118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:15:45.580118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:15:45.581276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:15:45.581276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:15:45.581276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:15:45.581276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:15:45.581276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:15:45.581276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:15:45.581276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:15:45.581276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:15:45.581276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:15:45.581276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:15:45.581276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:15:45.581276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:15:45.581276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:15:45.581276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:15:45.581276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:15:45.581276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:15:45.581276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:15:45.582555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:15:45.582555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:15:45.582555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:15:45.582555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:15:45.582555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 609/1203] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:15:45.587454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ca0d7-f807-4a10-9965-82937e500ebe None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:45.686195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ca0d7-f807-4a10-9965-82937e500ebe None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:45.714941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:45.720509 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27225d2-ae1d-486f-a497-02c42139543f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:45.722514 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27225d2-ae1d-486f-a497-02c42139543f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:45.722514 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27225d2-ae1d-486f-a497-02c42139543f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:45.722514 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27225d2-ae1d-486f-a497-02c42139543f None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:45.722514 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27225d2-ae1d-486f-a497-02c42139543f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:45.722961 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27225d2-ae1d-486f-a497-02c42139543f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:45.724133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27225d2-ae1d-486f-a497-02c42139543f None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:45.735049 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 595/1204] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:15:45.740777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81bfd1-7119-46d6-88e9-5938ac884abb None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:45.792406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:45.838173 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81bfd1-7119-46d6-88e9-5938ac884abb None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:45.869745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c36d4d-d614-4561-a817-fe79960ad1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:45.870424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c36d4d-d614-4561-a817-fe79960ad1d0 None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:45.871205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c36d4d-d614-4561-a817-fe79960ad1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:45.871757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c36d4d-d614-4561-a817-fe79960ad1d0 None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:45.872474 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c36d4d-d614-4561-a817-fe79960ad1d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:45.873883 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c36d4d-d614-4561-a817-fe79960ad1d0 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:45.999276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 610/1205] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Jul 20 15:15:45 2022] POST /identity/v3/users => generated 251 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:15:48.034228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb65ad6-249b-420d-a1ef-e1cb172803ea None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:48.034799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb65ad6-249b-420d-a1ef-e1cb172803ea None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:48.034799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb65ad6-249b-420d-a1ef-e1cb172803ea None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:48.036185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 596/1206] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:15:48.041436 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8d6ae-8e8b-45a5-aea7-e290585ebcaa None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:48.041857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8d6ae-8e8b-45a5-aea7-e290585ebcaa None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:48.041995 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8d6ae-8e8b-45a5-aea7-e290585ebcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:48.058936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-72c8d6ae-8e8b-45a5-aea7-e290585ebcaa None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:15:48.060118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-72c8d6ae-8e8b-45a5-aea7-e290585ebcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:15:48.127734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 611/1207] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:15:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:15:48.151982 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb65ad6-249b-420d-a1ef-e1cb172803ea None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:48.252933 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb65ad6-249b-420d-a1ef-e1cb172803ea None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:48.287197 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63872fb-d505-4be8-9535-798cfd440a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:48.287928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63872fb-d505-4be8-9535-798cfd440a41 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:48.288204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63872fb-d505-4be8-9535-798cfd440a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:48.288475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63872fb-d505-4be8-9535-798cfd440a41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:48.299038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63872fb-d505-4be8-9535-798cfd440a41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:15:48.306699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-f63872fb-d505-4be8-9535-798cfd440a41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:15:48.306699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:15:48.306699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:15:48.306699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:15:48.306699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:15:48.306699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:15:48.306699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:15:48.306699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:15:48.306699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:15:48.306699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:15:48.306699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:15:48.306699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:15:48.306699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:15:48.306699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:15:48.306699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:15:48.306699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:15:48.310369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:15:48.310369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:15:48.310369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:15:48.310369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:15:48.310369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:15:48.310369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:15:48.310369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:15:48.310369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:15:48.310369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:15:48.310369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:15:48.310369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:15:48.310369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:15:48.310369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:15:48.310369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:15:48.310369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:15:48.310369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:15:48.311277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:15:48.311277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:15:48.311277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:15:48.311277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:15:48.311277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:15:48.311277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:15:48.311277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:15:48.311277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:15:48.311277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:15:48.311277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:15:48.311277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:15:48.311277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:15:48.311277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 597/1208] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:15:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:15:48.314657 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8d6ae-8e8b-45a5-aea7-e290585ebcaa None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:48.352840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:48.409739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8d6ae-8e8b-45a5-aea7-e290585ebcaa None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:48.442587 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83594060-baf9-4cb0-a7d8-7cdb671e5a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:48.443262 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83594060-baf9-4cb0-a7d8-7cdb671e5a00 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:48.443749 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83594060-baf9-4cb0-a7d8-7cdb671e5a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:48.444268 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83594060-baf9-4cb0-a7d8-7cdb671e5a00 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:48.445420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83594060-baf9-4cb0-a7d8-7cdb671e5a00 None admin] RBAC: Adding query filter params (name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:48.445790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83594060-baf9-4cb0-a7d8-7cdb671e5a00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:48.447202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83594060-baf9-4cb0-a7d8-7cdb671e5a00 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:48.454979 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 612/1209] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:15:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:15:48.460669 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63872fb-d505-4be8-9535-798cfd440a41 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:48.512090 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:48.560794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63872fb-d505-4be8-9535-798cfd440a41 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:48.593050 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d45af-d8c1-4f69-8453-1ad5b03f6c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:48.596779 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d45af-d8c1-4f69-8453-1ad5b03f6c46 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:48.596779 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d45af-d8c1-4f69-8453-1ad5b03f6c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:48.596779 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d45af-d8c1-4f69-8453-1ad5b03f6c46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:48.605481 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d45af-d8c1-4f69-8453-1ad5b03f6c46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:48.607005 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d45af-d8c1-4f69-8453-1ad5b03f6c46 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:48.624672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-dc0d45af-d8c1-4f69-8453-1ad5b03f6c46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:15:48.624672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:15:48.624672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:15:48.624672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:15:48.624672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:15:48.624672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:15:48.624672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:15:48.624672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:15:48.624672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:15:48.624672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:15:48.624672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:15:48.624672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:15:48.624672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:15:48.624672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:15:48.624672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:15:48.625720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:15:48.625720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:15:48.625720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:15:48.625720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:15:48.625720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:15:48.625720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:15:48.625720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:15:48.625720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:15:48.625720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:15:48.625720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:15:48.625720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:15:48.625720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:15:48.625720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:15:48.625720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:15:48.625720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:15:48.625720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:15:48.625720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:15:48.626754 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:15:48.626754 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:15:48.626754 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:15:48.626754 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:15:48.626754 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 598/1210] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:15:48.631768 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83594060-baf9-4cb0-a7d8-7cdb671e5a00 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:48.726522 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:48.729266 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83594060-baf9-4cb0-a7d8-7cdb671e5a00 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:48.761731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc46848-7966-4ea3-8a69-6d7edfe0b606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:48.762420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc46848-7966-4ea3-8a69-6d7edfe0b606 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:48.762698 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc46848-7966-4ea3-8a69-6d7edfe0b606 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:48.762974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc46848-7966-4ea3-8a69-6d7edfe0b606 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:48.763688 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc46848-7966-4ea3-8a69-6d7edfe0b606 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:48.763995 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc46848-7966-4ea3-8a69-6d7edfe0b606 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:48.765371 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc46848-7966-4ea3-8a69-6d7edfe0b606 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:48.776146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 613/1211] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:15:48.782469 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d45af-d8c1-4f69-8453-1ad5b03f6c46 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:48.879576 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d45af-d8c1-4f69-8453-1ad5b03f6c46 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:48.881007 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:48.912636 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35377cf5-aba0-4cac-8920-231e64a98550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:48.913423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35377cf5-aba0-4cac-8920-231e64a98550 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:48.913714 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35377cf5-aba0-4cac-8920-231e64a98550 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:48.913991 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35377cf5-aba0-4cac-8920-231e64a98550 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:48.921325 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35377cf5-aba0-4cac-8920-231e64a98550 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:48.922787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35377cf5-aba0-4cac-8920-231e64a98550 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:48.937195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-35377cf5-aba0-4cac-8920-231e64a98550 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:15:48.937195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:15:48.937195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:15:48.937195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:15:48.937195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:15:48.937195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:15:48.937195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:15:48.937195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:15:48.937195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:15:48.937195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:15:48.937195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:15:48.937195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:15:48.937195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:15:48.937195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:15:48.937195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:15:48.937195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:15:48.938343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:15:48.939365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:15:48.939365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:15:48.939365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:15:48.939365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:15:48.939365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:15:48.939365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:15:48.939365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:15:48.939365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:15:48.939365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:15:48.939365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:15:48.939365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 599/1212] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:15:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:15:48.944187 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc46848-7966-4ea3-8a69-6d7edfe0b606 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:49.045446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc46848-7966-4ea3-8a69-6d7edfe0b606 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:49.078276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93729e3-abee-4150-9dab-84ad462bd92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:49.078955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93729e3-abee-4150-9dab-84ad462bd92d None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:49.079242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93729e3-abee-4150-9dab-84ad462bd92d None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:49.079537 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93729e3-abee-4150-9dab-84ad462bd92d None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:49.080348 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93729e3-abee-4150-9dab-84ad462bd92d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:49.080662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93729e3-abee-4150-9dab-84ad462bd92d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:49.082085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93729e3-abee-4150-9dab-84ad462bd92d None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:49.101622 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 614/1213] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Jul 20 15:15:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:15:49.107554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35377cf5-aba0-4cac-8920-231e64a98550 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:49.207245 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35377cf5-aba0-4cac-8920-231e64a98550 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:49.240272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cbe19d-ed95-41a6-b896-efc6c2c31ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:49.240913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbe19d-ed95-41a6-b896-efc6c2c31ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:49.241199 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbe19d-ed95-41a6-b896-efc6c2c31ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:49.241480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbe19d-ed95-41a6-b896-efc6c2c31ec2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:49.271354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:49.308984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cbe19d-ed95-41a6-b896-efc6c2c31ec2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:49.310480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cbe19d-ed95-41a6-b896-efc6c2c31ec2 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:49.408554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 600/1214] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:15:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:15:49.415164 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93729e3-abee-4150-9dab-84ad462bd92d None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:49.433998 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:49.517890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93729e3-abee-4150-9dab-84ad462bd92d None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:49.551013 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c4433-0724-4bc0-9d50-435d53b9b4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:49.551803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c4433-0724-4bc0-9d50-435d53b9b4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:49.552083 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c4433-0724-4bc0-9d50-435d53b9b4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:49.552365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c4433-0724-4bc0-9d50-435d53b9b4a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:49.562021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c4433-0724-4bc0-9d50-435d53b9b4a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:49.563491 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c4433-0724-4bc0-9d50-435d53b9b4a1 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:49.580310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-ba0c4433-0724-4bc0-9d50-435d53b9b4a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:15:49.580310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:15:49.580310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:15:49.580310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:15:49.580310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:15:49.580310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:15:49.580310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:15:49.580310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:15:49.580310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:15:49.580310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:15:49.580310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:15:49.580310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:15:49.580310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:15:49.580310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:15:49.580310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:15:49.584381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:15:49.584381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:15:49.584381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:15:49.584381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:15:49.584381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:15:49.584381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:15:49.584381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:15:49.584381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:15:49.584381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:15:49.584381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:15:49.584381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:15:49.584381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:15:49.584381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:15:49.584381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:15:49.584381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:15:49.584381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:15:49.584381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:15:49.585528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:15:49.585528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:15:49.585528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:15:49.585528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:15:49.585528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 615/1215] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:15:49.587224 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cbe19d-ed95-41a6-b896-efc6c2c31ec2 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:49.648561 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:49.687932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cbe19d-ed95-41a6-b896-efc6c2c31ec2 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:49.722574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f0390-d519-4bf9-8ae1-56d0ceb00a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:49.723282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f0390-d519-4bf9-8ae1-56d0ceb00a5f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:49.723604 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f0390-d519-4bf9-8ae1-56d0ceb00a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:49.724096 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f0390-d519-4bf9-8ae1-56d0ceb00a5f None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:49.724891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f0390-d519-4bf9-8ae1-56d0ceb00a5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:49.725199 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f0390-d519-4bf9-8ae1-56d0ceb00a5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:49.726764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f0390-d519-4bf9-8ae1-56d0ceb00a5f None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:49.740133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 601/1216] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:15:49.747742 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c4433-0724-4bc0-9d50-435d53b9b4a1 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:49.815640 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:49.845927 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c4433-0724-4bc0-9d50-435d53b9b4a1 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:49.877362 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6305c8f-531c-4170-9ff4-0955fc5ae846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:49.878142 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6305c8f-531c-4170-9ff4-0955fc5ae846 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:49.878452 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6305c8f-531c-4170-9ff4-0955fc5ae846 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:49.878726 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6305c8f-531c-4170-9ff4-0955fc5ae846 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:49.885054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6305c8f-531c-4170-9ff4-0955fc5ae846 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:49.887003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6305c8f-531c-4170-9ff4-0955fc5ae846 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:49.900731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-f6305c8f-531c-4170-9ff4-0955fc5ae846 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:15:49.900731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:15:49.900731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:15:49.900731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:15:49.900731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:15:49.900731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:15:49.900731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:15:49.900731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:15:49.900731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:15:49.900731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:15:49.900731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:15:49.900731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:15:49.900731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:15:49.900731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:15:49.900731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:15:49.902456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:15:49.902456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:15:49.902456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:15:49.902456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:15:49.902456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:15:49.902456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:15:49.902456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:15:49.902456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:15:49.902456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:15:49.902456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:15:49.902456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:15:49.902456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:15:49.902456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:15:49.902456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:15:49.902456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:15:49.902456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:15:49.902456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:15:49.903466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:15:49.903466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:15:49.903466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:15:49.903466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:15:49.903466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:15:49.903466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:15:49.903466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:15:49.903466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:15:49.903466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 616/1217] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:15:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:15:49.908499 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f0390-d519-4bf9-8ae1-56d0ceb00a5f None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:50.006400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f0390-d519-4bf9-8ae1-56d0ceb00a5f None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:50.029417 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:50.040580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf225294-60b2-4a11-a729-08ba3a9b4635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:50.041243 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225294-60b2-4a11-a729-08ba3a9b4635 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:50.041528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225294-60b2-4a11-a729-08ba3a9b4635 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:50.041808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225294-60b2-4a11-a729-08ba3a9b4635 None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:50.042670 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf225294-60b2-4a11-a729-08ba3a9b4635 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:50.042986 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf225294-60b2-4a11-a729-08ba3a9b4635 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:50.044579 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf225294-60b2-4a11-a729-08ba3a9b4635 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:50.058554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 602/1218] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:15:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:15:50.065139 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6305c8f-531c-4170-9ff4-0955fc5ae846 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:50.170251 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6305c8f-531c-4170-9ff4-0955fc5ae846 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:50.197846 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:50.204115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811aa51f-52a5-4eb1-9da1-3ea499418f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:50.204771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811aa51f-52a5-4eb1-9da1-3ea499418f3e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:50.205054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811aa51f-52a5-4eb1-9da1-3ea499418f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:50.205333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811aa51f-52a5-4eb1-9da1-3ea499418f3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:50.206151 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811aa51f-52a5-4eb1-9da1-3ea499418f3e None admin] RBAC: Adding query filter params (role.id=d820b1d3fa4144aca575c04645c099bc, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=2192547babde4449a46f9cb37ed64524) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:50.206495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811aa51f-52a5-4eb1-9da1-3ea499418f3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:50.207922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811aa51f-52a5-4eb1-9da1-3ea499418f3e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:50.213146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 617/1219] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:15:50 2022] GET /identity/v3/role_assignments?user.id=2192547babde4449a46f9cb37ed64524&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=d820b1d3fa4144aca575c04645c099bc => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:15:52.204690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4c27a-868d-46c3-9dd0-62d2825a57a8 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:52.205246 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4c27a-868d-46c3-9dd0-62d2825a57a8 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:52.205246 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4c27a-868d-46c3-9dd0-62d2825a57a8 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:52.206642 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 603/1220] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:15:52.212041 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ef661-e738-4137-8a6d-ffce5de0b95a None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:52.212497 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ef661-e738-4137-8a6d-ffce5de0b95a None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:52.212615 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ef661-e738-4137-8a6d-ffce5de0b95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:52.228996 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-0d0ef661-e738-4137-8a6d-ffce5de0b95a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:15:52.231575 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0d0ef661-e738-4137-8a6d-ffce5de0b95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:15:52.295553 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 618/1221] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:15:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:15:52.318217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4c27a-868d-46c3-9dd0-62d2825a57a8 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:52.419338 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4c27a-868d-46c3-9dd0-62d2825a57a8 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:52.453291 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dc54c-fbbf-45ed-a694-7e8f6d3f813f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:52.454116 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dc54c-fbbf-45ed-a694-7e8f6d3f813f None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:52.454426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dc54c-fbbf-45ed-a694-7e8f6d3f813f None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:52.454701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dc54c-fbbf-45ed-a694-7e8f6d3f813f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:52.465210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9dc54c-fbbf-45ed-a694-7e8f6d3f813f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:15:52.472898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-4e9dc54c-fbbf-45ed-a694-7e8f6d3f813f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:15:52.472898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:15:52.472898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:15:52.472898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:15:52.472898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:15:52.472898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:15:52.472898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:15:52.472898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:15:52.472898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:15:52.472898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:15:52.472898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:15:52.472898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:15:52.472898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:15:52.472898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:15:52.472898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:15:52.472898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:15:52.476958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:15:52.476958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:15:52.476958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:15:52.476958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:15:52.476958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:15:52.476958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:15:52.476958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:15:52.476958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:15:52.476958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:15:52.476958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:15:52.476958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:15:52.476958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:15:52.476958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:15:52.476958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:15:52.476958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:15:52.476958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:15:52.478042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:15:52.478042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:15:52.478042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:15:52.478042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:15:52.478042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:15:52.478042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:15:52.478042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:15:52.478042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:15:52.478042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:15:52.478042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:15:52.478042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:15:52.478042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:15:52.478042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 604/1222] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:15:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:15:52.480866 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ef661-e738-4137-8a6d-ffce5de0b95a None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:52.576592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ef661-e738-4137-8a6d-ffce5de0b95a None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:52.600037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:52.610863 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4efdd1b-e8f3-402d-a07c-812f0f911e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:52.611525 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efdd1b-e8f3-402d-a07c-812f0f911e04 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:52.611808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efdd1b-e8f3-402d-a07c-812f0f911e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:52.612082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efdd1b-e8f3-402d-a07c-812f0f911e04 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:52.612815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4efdd1b-e8f3-402d-a07c-812f0f911e04 None admin] RBAC: Adding query filter params (name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:52.613119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4efdd1b-e8f3-402d-a07c-812f0f911e04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:52.614526 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4efdd1b-e8f3-402d-a07c-812f0f911e04 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:52.622549 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 619/1223] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:15:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:15:52.628254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dc54c-fbbf-45ed-a694-7e8f6d3f813f None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:52.673839 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:52.728396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dc54c-fbbf-45ed-a694-7e8f6d3f813f None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:52.762156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079e957-5944-450e-a94b-27a813e14163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:52.763024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079e957-5944-450e-a94b-27a813e14163 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:52.763363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079e957-5944-450e-a94b-27a813e14163 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:52.763648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079e957-5944-450e-a94b-27a813e14163 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:52.774620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079e957-5944-450e-a94b-27a813e14163 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:52.786307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079e957-5944-450e-a94b-27a813e14163 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:52.795134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-0079e957-5944-450e-a94b-27a813e14163 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:15:52.795134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:15:52.795134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:15:52.795134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:15:52.795134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:15:52.795134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:15:52.795134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:15:52.795134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:15:52.795134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:15:52.795134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:15:52.795134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:15:52.795134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:15:52.795134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:15:52.795134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:15:52.795134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:15:52.796195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:15:52.796195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:15:52.796195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:15:52.796195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:15:52.796195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:15:52.796195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:15:52.796195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:15:52.796195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:15:52.796195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:15:52.796195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:15:52.796195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:15:52.796195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:15:52.796195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:15:52.796195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:15:52.796195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:15:52.796195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:15:52.796195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:15:52.797351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:15:52.797351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:15:52.797351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:15:52.797351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:15:52.797351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 605/1224] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:15:52.802434 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4efdd1b-e8f3-402d-a07c-812f0f911e04 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:52.903639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4efdd1b-e8f3-402d-a07c-812f0f911e04 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:52.935444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a2fff-2cac-4cd8-ba55-a01d92bf3a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:52.936042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a2fff-2cac-4cd8-ba55-a01d92bf3a11 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:52.936324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a2fff-2cac-4cd8-ba55-a01d92bf3a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:52.936600 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a2fff-2cac-4cd8-ba55-a01d92bf3a11 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:52.937312 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a2fff-2cac-4cd8-ba55-a01d92bf3a11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:52.937617 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a2fff-2cac-4cd8-ba55-a01d92bf3a11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:52.939019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a2fff-2cac-4cd8-ba55-a01d92bf3a11 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:52.950920 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 620/1225] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:15:52.956690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079e957-5944-450e-a94b-27a813e14163 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:53.047598 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:53.054086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079e957-5944-450e-a94b-27a813e14163 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:53.086706 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f1f59-57aa-429d-b195-dea4c6f9f314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:53.087526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f1f59-57aa-429d-b195-dea4c6f9f314 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:53.087808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f1f59-57aa-429d-b195-dea4c6f9f314 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:53.088111 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f1f59-57aa-429d-b195-dea4c6f9f314 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:53.096250 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f1f59-57aa-429d-b195-dea4c6f9f314 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:53.097691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f1f59-57aa-429d-b195-dea4c6f9f314 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:53.112730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-840f1f59-57aa-429d-b195-dea4c6f9f314 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:15:53.112730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:15:53.112730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:15:53.112730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:15:53.112730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:15:53.112730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:15:53.112730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:15:53.112730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:15:53.112730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:15:53.112730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:15:53.112730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:15:53.112730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:15:53.112730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:15:53.112730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:15:53.112730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:15:53.112730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:15:53.113891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:15:53.114922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:15:53.114922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:15:53.114922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:15:53.114922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:15:53.114922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:15:53.114922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:15:53.114922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:15:53.114922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:15:53.114922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:15:53.114922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:15:53.115887 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 606/1226] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:15:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:15:53.119934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a2fff-2cac-4cd8-ba55-a01d92bf3a11 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:53.165699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:53.216881 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a2fff-2cac-4cd8-ba55-a01d92bf3a11 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:53.249970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827ee95-126d-4a89-8bf2-ba2828cbcf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:53.250660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827ee95-126d-4a89-8bf2-ba2828cbcf0e None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:53.250942 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827ee95-126d-4a89-8bf2-ba2828cbcf0e None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:53.251215 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827ee95-126d-4a89-8bf2-ba2828cbcf0e None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:53.252038 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827ee95-126d-4a89-8bf2-ba2828cbcf0e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:53.252346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827ee95-126d-4a89-8bf2-ba2828cbcf0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:53.253791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827ee95-126d-4a89-8bf2-ba2828cbcf0e None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:53.270652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 621/1227] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Jul 20 15:15:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:15:53.276430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f1f59-57aa-429d-b195-dea4c6f9f314 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:53.378314 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f1f59-57aa-429d-b195-dea4c6f9f314 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:53.412606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2439c-bfe9-4448-ad0a-07f3c5a1a10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:53.413389 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2439c-bfe9-4448-ad0a-07f3c5a1a10d None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:53.413672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2439c-bfe9-4448-ad0a-07f3c5a1a10d None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:53.413945 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2439c-bfe9-4448-ad0a-07f3c5a1a10d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:53.423795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf2439c-bfe9-4448-ad0a-07f3c5a1a10d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:53.425227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf2439c-bfe9-4448-ad0a-07f3c5a1a10d None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:53.442956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-6cf2439c-bfe9-4448-ad0a-07f3c5a1a10d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:15:53.442956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:15:53.442956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:15:53.442956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:15:53.442956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:15:53.442956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:15:53.442956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:15:53.442956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:15:53.442956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:15:53.442956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:15:53.442956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:15:53.442956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:15:53.442956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:15:53.442956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:15:53.442956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:15:53.448283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:15:53.448283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:15:53.448283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:15:53.448283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:15:53.448283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:15:53.448283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:15:53.448283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:15:53.448283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:15:53.448283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:15:53.448283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:15:53.448283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:15:53.448283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:15:53.448283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:15:53.448283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:15:53.448283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:15:53.448283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:15:53.448283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:15:53.449937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:15:53.449937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:15:53.449937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:15:53.449937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:15:53.449937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 607/1228] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:15:53.450692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827ee95-126d-4a89-8bf2-ba2828cbcf0e None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:53.548435 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827ee95-126d-4a89-8bf2-ba2828cbcf0e None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:53.550842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:53.581296 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4675e-8110-46a4-abb5-c9ddbd98da9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:53.581960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4675e-8110-46a4-abb5-c9ddbd98da9a None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:53.582244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4675e-8110-46a4-abb5-c9ddbd98da9a None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:53.582549 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4675e-8110-46a4-abb5-c9ddbd98da9a None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:53.583267 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4675e-8110-46a4-abb5-c9ddbd98da9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:53.583574 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4675e-8110-46a4-abb5-c9ddbd98da9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:53.584948 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4675e-8110-46a4-abb5-c9ddbd98da9a None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:53.596133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 622/1229] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:15:53.601800 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2439c-bfe9-4448-ad0a-07f3c5a1a10d None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:53.611042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:53.698243 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2439c-bfe9-4448-ad0a-07f3c5a1a10d None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:53.729633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b7e40-c7de-447a-9231-a102f79bffef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:53.730443 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b7e40-c7de-447a-9231-a102f79bffef None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:53.730723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b7e40-c7de-447a-9231-a102f79bffef None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:53.730993 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b7e40-c7de-447a-9231-a102f79bffef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:53.737846 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b7e40-c7de-447a-9231-a102f79bffef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:53.739303 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b7e40-c7de-447a-9231-a102f79bffef None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:53.754034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-5a6b7e40-c7de-447a-9231-a102f79bffef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:15:53.754034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:15:53.754034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:15:53.754034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:15:53.754034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:15:53.754034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:15:53.754034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:15:53.754034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:15:53.754034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:15:53.754034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:15:53.754034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:15:53.754034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:15:53.754034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:15:53.754034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:15:53.754034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:15:53.755146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:15:53.755146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:15:53.755146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:15:53.755146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:15:53.755146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:15:53.755146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:15:53.755146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:15:53.755146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:15:53.755146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:15:53.755146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:15:53.755146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:15:53.755146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:15:53.755146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:15:53.755146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:15:53.755146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:15:53.755146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:15:53.755146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:15:53.756320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:15:53.756320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:15:53.756320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:15:53.756320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:15:53.756320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:15:53.756320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:15:53.756320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:15:53.756320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:15:53.756320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 608/1230] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:15:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:15:53.760988 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4675e-8110-46a4-abb5-c9ddbd98da9a None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:53.858889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4675e-8110-46a4-abb5-c9ddbd98da9a None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:53.890690 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa1c2e-0a61-43eb-8b05-948812e7cf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:53.891340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa1c2e-0a61-43eb-8b05-948812e7cf1d None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:53.891621 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa1c2e-0a61-43eb-8b05-948812e7cf1d None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:53.891894 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa1c2e-0a61-43eb-8b05-948812e7cf1d None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:53.892663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa1c2e-0a61-43eb-8b05-948812e7cf1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:53.892969 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa1c2e-0a61-43eb-8b05-948812e7cf1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:53.894493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa1c2e-0a61-43eb-8b05-948812e7cf1d None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:53.907083 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 623/1231] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:15:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:15:53.913098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b7e40-c7de-447a-9231-a102f79bffef None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:53.978214 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:54.012032 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b7e40-c7de-447a-9231-a102f79bffef None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:54.046284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e13a1-5532-4c91-b98c-3b234e0fe6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:54.047179 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e13a1-5532-4c91-b98c-3b234e0fe6d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:54.047463 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e13a1-5532-4c91-b98c-3b234e0fe6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:54.047738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e13a1-5532-4c91-b98c-3b234e0fe6d0 None admin] PATH_INFO: `/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/2192547babde4449a46f9cb37ed64524/roles/d820b1d3fa4144aca575c04645c099bc` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:54.061997 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e13a1-5532-4c91-b98c-3b234e0fe6d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdae1cb71dd449b8a18062b7b52889e4, user_id=2192547babde4449a46f9cb37ed64524, role_id=d820b1d3fa4144aca575c04645c099bc)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:54.063482 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e13a1-5532-4c91-b98c-3b234e0fe6d0 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:54.216364 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 609/1232] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:15:53 2022] PUT /identity/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/2192547babde4449a46f9cb37ed64524/roles/d820b1d3fa4144aca575c04645c099bc => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:15:56.234761 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10777fc-958b-4920-84fb-2abf102a999d None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:56.235364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10777fc-958b-4920-84fb-2abf102a999d None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:56.235459 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10777fc-958b-4920-84fb-2abf102a999d None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:56.236797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 624/1233] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:15:56.242045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730886b-cbbf-471a-a07e-be5653b36429 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:56.242493 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730886b-cbbf-471a-a07e-be5653b36429 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:56.242658 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730886b-cbbf-471a-a07e-be5653b36429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:56.259286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-f730886b-cbbf-471a-a07e-be5653b36429 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:15:56.260466 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f730886b-cbbf-471a-a07e-be5653b36429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:15:56.325712 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 610/1234] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:15:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:15:56.349364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10777fc-958b-4920-84fb-2abf102a999d None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:56.375419 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:56.449173 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10777fc-958b-4920-84fb-2abf102a999d None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:56.485523 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a7f21-2ad8-44b0-b33f-921319e87072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:56.486331 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a7f21-2ad8-44b0-b33f-921319e87072 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:56.486612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a7f21-2ad8-44b0-b33f-921319e87072 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:56.486884 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a7f21-2ad8-44b0-b33f-921319e87072 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:56.496783 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a7f21-2ad8-44b0-b33f-921319e87072 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:15:56.504369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-703a7f21-2ad8-44b0-b33f-921319e87072 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:15:56.504369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:15:56.504369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:15:56.504369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:15:56.504369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:15:56.504369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:15:56.504369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:15:56.504369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:15:56.504369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:15:56.504369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:15:56.504369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:15:56.504369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:15:56.504369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:15:56.504369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:15:56.504369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:15:56.504369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:15:56.505545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:15:56.505545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:15:56.505545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:15:56.505545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:15:56.505545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:15:56.505545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:15:56.505545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:15:56.505545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:15:56.505545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:15:56.505545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:15:56.505545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:15:56.505545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:15:56.505545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:15:56.505545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:15:56.505545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:15:56.505545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:15:56.506658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:15:56.506658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:15:56.506658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:15:56.506658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:15:56.506658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:15:56.506658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:15:56.506658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:15:56.506658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:15:56.506658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:15:56.506658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:15:56.506658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 15:15:56.506658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:15:56.506658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 625/1235] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Jul 20 15:15:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:15:56.511987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730886b-cbbf-471a-a07e-be5653b36429 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:56.557963 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:56.608768 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730886b-cbbf-471a-a07e-be5653b36429 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:56.642118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414e0e5-0eb5-45f7-87ca-730f770e5704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:56.642807 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414e0e5-0eb5-45f7-87ca-730f770e5704 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:56.643089 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414e0e5-0eb5-45f7-87ca-730f770e5704 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:56.646473 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414e0e5-0eb5-45f7-87ca-730f770e5704 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:56.646473 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414e0e5-0eb5-45f7-87ca-730f770e5704 None admin] RBAC: Adding query filter params (name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:56.646473 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414e0e5-0eb5-45f7-87ca-730f770e5704 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:56.646473 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414e0e5-0eb5-45f7-87ca-730f770e5704 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:56.653519 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 611/1236] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Jul 20 15:15:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:15:56.659708 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a7f21-2ad8-44b0-b33f-921319e87072 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:56.741983 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:56.757241 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a7f21-2ad8-44b0-b33f-921319e87072 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:56.791545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55727835-f038-4041-9767-b7edd046686d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:56.792328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55727835-f038-4041-9767-b7edd046686d None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:56.792608 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55727835-f038-4041-9767-b7edd046686d None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:56.792882 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55727835-f038-4041-9767-b7edd046686d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:56.803106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55727835-f038-4041-9767-b7edd046686d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:56.804529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55727835-f038-4041-9767-b7edd046686d None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:56.822128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-55727835-f038-4041-9767-b7edd046686d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:15:56.822128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:15:56.822128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:15:56.822128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:15:56.822128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:15:56.822128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:15:56.822128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:15:56.822128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:15:56.822128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:15:56.822128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:15:56.822128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:15:56.822128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:15:56.822128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:15:56.822128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:15:56.822128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:15:56.823063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:15:56.823063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:15:56.823063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:15:56.823063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:15:56.823063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:15:56.823063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:15:56.823063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:15:56.823063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:15:56.823063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:15:56.823063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:15:56.823063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:15:56.823063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:15:56.823063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:15:56.823063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:15:56.823063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:15:56.823063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:15:56.823063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:15:56.824143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:15:56.824143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:15:56.824143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:15:56.824143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:15:56.824143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 626/1237] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:15:56.829602 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414e0e5-0eb5-45f7-87ca-730f770e5704 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:56.926932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414e0e5-0eb5-45f7-87ca-730f770e5704 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:56.936056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:56.960406 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f714218-f431-4f17-8d9c-ea5a11cc2ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:56.961060 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f714218-f431-4f17-8d9c-ea5a11cc2ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:56.961342 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f714218-f431-4f17-8d9c-ea5a11cc2ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:56.961620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f714218-f431-4f17-8d9c-ea5a11cc2ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:56.962456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f714218-f431-4f17-8d9c-ea5a11cc2ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:56.962762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f714218-f431-4f17-8d9c-ea5a11cc2ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:56.964113 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f714218-f431-4f17-8d9c-ea5a11cc2ad8 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:56.975525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 612/1238] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:15:56.981217 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55727835-f038-4041-9767-b7edd046686d None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:57.078503 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55727835-f038-4041-9767-b7edd046686d None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:57.113833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8e033-d10f-4df0-9302-9c0debb3497b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:57.114759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8e033-d10f-4df0-9302-9c0debb3497b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:57.115046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8e033-d10f-4df0-9302-9c0debb3497b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:57.115324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8e033-d10f-4df0-9302-9c0debb3497b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:57.118004 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:57.123529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8e033-d10f-4df0-9302-9c0debb3497b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:57.124970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8e033-d10f-4df0-9302-9c0debb3497b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:57.140403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-03c8e033-d10f-4df0-9302-9c0debb3497b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:15:57.140403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:15:57.140403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:15:57.140403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:15:57.140403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:15:57.140403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:15:57.140403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:15:57.140403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:15:57.140403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:15:57.140403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:15:57.140403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:15:57.140403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:15:57.140403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:15:57.140403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:15:57.140403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:15:57.140403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:15:57.141540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:15:57.142806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:15:57.142806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:15:57.142806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:15:57.142806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:15:57.142806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:15:57.142806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:15:57.142806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:15:57.142806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:15:57.142806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:15:57.142806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:15:57.142806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 627/1239] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:15:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:15:57.147476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f714218-f431-4f17-8d9c-ea5a11cc2ad8 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:57.244412 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f714218-f431-4f17-8d9c-ea5a11cc2ad8 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:57.279075 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc76ec-4923-4569-85bd-bfa9612d8bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:57.279744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc76ec-4923-4569-85bd-bfa9612d8bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:57.280031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc76ec-4923-4569-85bd-bfa9612d8bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:57.280312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc76ec-4923-4569-85bd-bfa9612d8bc1 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:57.281144 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc76ec-4923-4569-85bd-bfa9612d8bc1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:57.281455 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc76ec-4923-4569-85bd-bfa9612d8bc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:57.282979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc76ec-4923-4569-85bd-bfa9612d8bc1 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:57.303232 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 613/1240] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Jul 20 15:15:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:15:57.309118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8e033-d10f-4df0-9302-9c0debb3497b None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:57.407715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8e033-d10f-4df0-9302-9c0debb3497b None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:57.440441 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60991d10-e6ab-4802-8d68-d232ddc6808d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:57.441085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60991d10-e6ab-4802-8d68-d232ddc6808d None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:57.441364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60991d10-e6ab-4802-8d68-d232ddc6808d None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:57.441638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60991d10-e6ab-4802-8d68-d232ddc6808d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:57.500392 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:57.506620 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60991d10-e6ab-4802-8d68-d232ddc6808d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:57.508062 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60991d10-e6ab-4802-8d68-d232ddc6808d None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:57.602215 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 628/1241] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:15:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:15:57.608066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc76ec-4923-4569-85bd-bfa9612d8bc1 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:57.635238 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:57.705430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc76ec-4923-4569-85bd-bfa9612d8bc1 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:57.739150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b7da1-7970-4926-ac0f-be3932c9ea6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:57.739933 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b7da1-7970-4926-ac0f-be3932c9ea6b None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:57.740241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b7da1-7970-4926-ac0f-be3932c9ea6b None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:57.740511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b7da1-7970-4926-ac0f-be3932c9ea6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:57.751249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b7da1-7970-4926-ac0f-be3932c9ea6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:57.752635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b7da1-7970-4926-ac0f-be3932c9ea6b None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:57.770140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-540b7da1-7970-4926-ac0f-be3932c9ea6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:15:57.770140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:15:57.770140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:15:57.770140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:15:57.770140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:15:57.770140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:15:57.770140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:15:57.770140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:15:57.770140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:15:57.770140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:15:57.770140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:15:57.770140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:15:57.770140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:15:57.770140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:15:57.770140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:15:57.771254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:15:57.771254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:15:57.771254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:15:57.771254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:15:57.771254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:15:57.771254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:15:57.771254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:15:57.771254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:15:57.771254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:15:57.771254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:15:57.771254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:15:57.771254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:15:57.771254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:15:57.771254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:15:57.771254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:15:57.771254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:15:57.771254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:15:57.772614 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:15:57.772614 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:15:57.772614 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:15:57.772614 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:15:57.772614 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 614/1242] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:15:57.777143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60991d10-e6ab-4802-8d68-d232ddc6808d None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:57.869383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:57.875817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60991d10-e6ab-4802-8d68-d232ddc6808d None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:57.908450 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647e8e7-1b12-4836-ada4-82f4d0ddf090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:57.909102 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647e8e7-1b12-4836-ada4-82f4d0ddf090 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:57.909385 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647e8e7-1b12-4836-ada4-82f4d0ddf090 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:57.909663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647e8e7-1b12-4836-ada4-82f4d0ddf090 None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:57.910406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647e8e7-1b12-4836-ada4-82f4d0ddf090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:57.910708 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647e8e7-1b12-4836-ada4-82f4d0ddf090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:57.912243 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647e8e7-1b12-4836-ada4-82f4d0ddf090 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:57.923671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 629/1243] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:15:57.929408 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b7da1-7970-4926-ac0f-be3932c9ea6b None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:58.008120 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:58.028695 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b7da1-7970-4926-ac0f-be3932c9ea6b None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:58.063675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6991e-ee83-4a26-9700-f9ca044fe61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:58.064460 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6991e-ee83-4a26-9700-f9ca044fe61c None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:58.064746 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6991e-ee83-4a26-9700-f9ca044fe61c None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:58.065024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6991e-ee83-4a26-9700-f9ca044fe61c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:58.071814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6991e-ee83-4a26-9700-f9ca044fe61c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:58.073262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6991e-ee83-4a26-9700-f9ca044fe61c None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:58.087254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-30d6991e-ee83-4a26-9700-f9ca044fe61c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:15:58.087254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:15:58.087254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:15:58.087254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:15:58.087254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:15:58.087254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:15:58.087254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:15:58.087254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:15:58.087254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:15:58.087254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:15:58.087254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:15:58.087254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:15:58.087254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:15:58.087254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:15:58.087254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:15:58.088370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:15:58.088370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:15:58.088370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:15:58.088370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:15:58.088370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:15:58.088370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:15:58.088370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:15:58.088370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:15:58.088370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:15:58.088370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:15:58.088370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:15:58.088370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:15:58.088370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:15:58.088370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:15:58.088370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:15:58.088370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:15:58.088370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:15:58.089487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:15:58.089487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:15:58.089487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:15:58.089487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:15:58.089487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:15:58.089487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:15:58.089487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:15:58.089487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:15:58.089487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 615/1244] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:15:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:15:58.095160 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647e8e7-1b12-4836-ada4-82f4d0ddf090 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:58.194449 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647e8e7-1b12-4836-ada4-82f4d0ddf090 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:58.226651 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475eec7-9374-4b22-9ab1-9f164dc75a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:58.227313 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475eec7-9374-4b22-9ab1-9f164dc75a4a None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:58.227596 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475eec7-9374-4b22-9ab1-9f164dc75a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:58.227871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475eec7-9374-4b22-9ab1-9f164dc75a4a None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:58.228651 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475eec7-9374-4b22-9ab1-9f164dc75a4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:58.228959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475eec7-9374-4b22-9ab1-9f164dc75a4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:58.230428 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475eec7-9374-4b22-9ab1-9f164dc75a4a None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:58.242933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 630/1245] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:15:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:15:58.249765 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6991e-ee83-4a26-9700-f9ca044fe61c None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:15:58.347656 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6991e-ee83-4a26-9700-f9ca044fe61c None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:15:58.380149 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75761bc4-81c3-4821-bcd8-15c2573fd239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:15:58.380785 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75761bc4-81c3-4821-bcd8-15c2573fd239 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:15:58.381063 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75761bc4-81c3-4821-bcd8-15c2573fd239 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:15:58.381336 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75761bc4-81c3-4821-bcd8-15c2573fd239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:15:58.382158 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75761bc4-81c3-4821-bcd8-15c2573fd239 None admin] RBAC: Adding query filter params (role.id=d820b1d3fa4144aca575c04645c099bc, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=2192547babde4449a46f9cb37ed64524) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:15:58.382496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75761bc4-81c3-4821-bcd8-15c2573fd239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:15:58.383906 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75761bc4-81c3-4821-bcd8-15c2573fd239 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:15:58.389548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 616/1246] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:15:58 2022] GET /identity/v3/role_assignments?user.id=2192547babde4449a46f9cb37ed64524&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=d820b1d3fa4144aca575c04645c099bc => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:00.465789 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f90bc7-9f04-44f6-af84-492822bf3a15 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:00.466402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f90bc7-9f04-44f6-af84-492822bf3a15 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:00.466402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f90bc7-9f04-44f6-af84-492822bf3a15 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:00.468339 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 631/1247] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:16:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:16:00.473545 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5dd0f3-6d62-48a3-b50a-327335e5dbcc None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:00.473919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5dd0f3-6d62-48a3-b50a-327335e5dbcc None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:00.474100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5dd0f3-6d62-48a3-b50a-327335e5dbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:00.479173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:00.490736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-7b5dd0f3-6d62-48a3-b50a-327335e5dbcc None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:16:00.491920 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-7b5dd0f3-6d62-48a3-b50a-327335e5dbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:16:00.556356 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 617/1248] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:16:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:16:00.580637 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f90bc7-9f04-44f6-af84-492822bf3a15 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:00.601270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:00.682358 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f90bc7-9f04-44f6-af84-492822bf3a15 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:00.715375 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb625e-1229-41b8-94a8-120c2e40a866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:00.716200 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb625e-1229-41b8-94a8-120c2e40a866 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:00.716435 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb625e-1229-41b8-94a8-120c2e40a866 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:00.716770 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb625e-1229-41b8-94a8-120c2e40a866 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:00.726705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb625e-1229-41b8-94a8-120c2e40a866 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:16:00.734202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-bdfb625e-1229-41b8-94a8-120c2e40a866 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:16:00.734202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:00.734202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:00.734202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:00.734202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:00.734202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:00.734202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:00.734202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:00.734202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:00.734202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:00.734202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:00.734202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:00.734202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:16:00.734202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:16:00.734202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:16:00.734202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:16:00.735199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:16:00.735199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:16:00.735199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:16:00.735199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:16:00.735199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:00.735199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:00.735199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:00.735199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:00.735199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:00.735199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:00.735199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:00.735199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:00.735199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:00.735199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:00.735199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:00.735199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:00.737320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:00.737320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:00.737320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:00.737320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:00.737320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:16:00.737320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:16:00.737320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:16:00.737320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:16:00.737320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:16:00.737320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:16:00.737320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:16:00.737320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:16:00.737320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 632/1249] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:16:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:16:00.741592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5dd0f3-6d62-48a3-b50a-327335e5dbcc None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:00.839356 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5dd0f3-6d62-48a3-b50a-327335e5dbcc None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:00.869556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:00.873865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7c6dc-32c3-40e8-beb5-d770a2563be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:00.874606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7c6dc-32c3-40e8-beb5-d770a2563be2 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:00.874886 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7c6dc-32c3-40e8-beb5-d770a2563be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:00.875160 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7c6dc-32c3-40e8-beb5-d770a2563be2 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:00.875899 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7c6dc-32c3-40e8-beb5-d770a2563be2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:00.876212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7c6dc-32c3-40e8-beb5-d770a2563be2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:00.877558 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7c6dc-32c3-40e8-beb5-d770a2563be2 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:00.885583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 618/1250] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:16:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:00.892017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb625e-1229-41b8-94a8-120c2e40a866 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:00.968093 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:00.990577 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb625e-1229-41b8-94a8-120c2e40a866 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:01.023028 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c4c4f-3049-4571-8df4-a7e5ec477e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:01.023840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c4c4f-3049-4571-8df4-a7e5ec477e19 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:01.024184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c4c4f-3049-4571-8df4-a7e5ec477e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:01.024460 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c4c4f-3049-4571-8df4-a7e5ec477e19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:01.034770 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c4c4f-3049-4571-8df4-a7e5ec477e19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:01.036208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c4c4f-3049-4571-8df4-a7e5ec477e19 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:01.053310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-094c4c4f-3049-4571-8df4-a7e5ec477e19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:16:01.053310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:01.053310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:01.053310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:01.053310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:01.053310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:01.053310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:01.053310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:01.053310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:01.053310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:01.053310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:01.053310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:01.053310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:16:01.053310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:16:01.053310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:16:01.054424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:16:01.054424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:01.054424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:01.054424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:01.054424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:01.054424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:01.054424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:01.054424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:01.054424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:01.054424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:01.054424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:01.054424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:01.054424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:01.054424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:01.054424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:01.054424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:01.054424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:01.055552 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:16:01.055552 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:16:01.055552 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:16:01.055552 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:16:01.055552 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 633/1251] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:16:01.060855 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7c6dc-32c3-40e8-beb5-d770a2563be2 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:01.161352 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7c6dc-32c3-40e8-beb5-d770a2563be2 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:01.195530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55459310-983e-4e64-9a71-acf4c4b0c8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:01.196186 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55459310-983e-4e64-9a71-acf4c4b0c8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:01.196466 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55459310-983e-4e64-9a71-acf4c4b0c8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:01.196746 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55459310-983e-4e64-9a71-acf4c4b0c8b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:01.197464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55459310-983e-4e64-9a71-acf4c4b0c8b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:01.197772 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55459310-983e-4e64-9a71-acf4c4b0c8b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:01.199250 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55459310-983e-4e64-9a71-acf4c4b0c8b7 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:01.211140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 619/1252] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:01.216828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c4c4f-3049-4571-8df4-a7e5ec477e19 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:01.318411 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c4c4f-3049-4571-8df4-a7e5ec477e19 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:01.344100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:01.351889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4090150-ddee-4638-8d95-eae58e6448a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:01.352685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4090150-ddee-4638-8d95-eae58e6448a2 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:01.352972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4090150-ddee-4638-8d95-eae58e6448a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:01.353251 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4090150-ddee-4638-8d95-eae58e6448a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:01.361055 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4090150-ddee-4638-8d95-eae58e6448a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:01.362560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4090150-ddee-4638-8d95-eae58e6448a2 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:01.376953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-d4090150-ddee-4638-8d95-eae58e6448a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:16:01.376953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:01.376953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:01.376953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:01.376953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:01.376953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:01.376953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:01.376953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:01.376953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:01.376953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:01.376953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:01.376953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:01.376953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:16:01.376953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:16:01.376953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:16:01.376953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:01.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:01.379564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:01.379564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:01.379564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:16:01.379564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:16:01.379564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:16:01.379564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:16:01.379564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:16:01.379564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:16:01.379564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:16:01.379564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:16:01.379564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 634/1253] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:16:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:16:01.384197 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55459310-983e-4e64-9a71-acf4c4b0c8b7 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:01.438467 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:01.483336 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55459310-983e-4e64-9a71-acf4c4b0c8b7 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:01.515048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16e485-ce58-465b-9d37-70ee8a92c725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:01.515721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16e485-ce58-465b-9d37-70ee8a92c725 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:01.516046 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16e485-ce58-465b-9d37-70ee8a92c725 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:01.516321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16e485-ce58-465b-9d37-70ee8a92c725 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:01.517146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c16e485-ce58-465b-9d37-70ee8a92c725 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:01.517453 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c16e485-ce58-465b-9d37-70ee8a92c725 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:01.518889 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c16e485-ce58-465b-9d37-70ee8a92c725 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:01.536386 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 620/1254] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Jul 20 15:16:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:01.542200 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4090150-ddee-4638-8d95-eae58e6448a2 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:01.640943 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4090150-ddee-4638-8d95-eae58e6448a2 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:01.673848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f176cd-991e-41ca-b61b-942dbb029ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:01.674524 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f176cd-991e-41ca-b61b-942dbb029ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:01.674810 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f176cd-991e-41ca-b61b-942dbb029ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:01.675090 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f176cd-991e-41ca-b61b-942dbb029ca3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:01.728685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:01.740296 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f176cd-991e-41ca-b61b-942dbb029ca3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:01.741745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f176cd-991e-41ca-b61b-942dbb029ca3 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:01.839162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 635/1255] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:16:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:16:01.845664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16e485-ce58-465b-9d37-70ee8a92c725 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:01.941862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16e485-ce58-465b-9d37-70ee8a92c725 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:01.954813 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:01.975927 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586d20b-81d8-4716-994a-d8c87688c148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:01.976715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586d20b-81d8-4716-994a-d8c87688c148 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:01.976999 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586d20b-81d8-4716-994a-d8c87688c148 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:01.977286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586d20b-81d8-4716-994a-d8c87688c148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:01.987296 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586d20b-81d8-4716-994a-d8c87688c148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:01.988728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586d20b-81d8-4716-994a-d8c87688c148 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:02.006217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-7586d20b-81d8-4716-994a-d8c87688c148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:16:02.006217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:02.006217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:02.006217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:02.006217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:02.006217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:02.006217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:02.006217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:02.006217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:02.006217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:02.006217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:02.006217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:02.006217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:16:02.006217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:16:02.006217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:16:02.007357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:16:02.007357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:02.007357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:02.007357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:02.007357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:02.007357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:02.007357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:02.007357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:02.007357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:02.007357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:02.007357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:02.007357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:02.007357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:02.007357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:02.007357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:02.007357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:02.007357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:02.008505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:16:02.008505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:16:02.008505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:16:02.008505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:16:02.008505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 621/1256] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:16:02.014316 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f176cd-991e-41ca-b61b-942dbb029ca3 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:02.099839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:02.111654 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f176cd-991e-41ca-b61b-942dbb029ca3 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:02.145537 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3ef46-3366-4b98-8886-a96c89d7ab6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:02.146196 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3ef46-3366-4b98-8886-a96c89d7ab6b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:02.146511 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3ef46-3366-4b98-8886-a96c89d7ab6b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:02.146795 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3ef46-3366-4b98-8886-a96c89d7ab6b None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:02.147514 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3ef46-3366-4b98-8886-a96c89d7ab6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:02.147823 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3ef46-3366-4b98-8886-a96c89d7ab6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:02.149193 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3ef46-3366-4b98-8886-a96c89d7ab6b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:02.160179 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 636/1257] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:02.165946 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586d20b-81d8-4716-994a-d8c87688c148 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:02.262609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586d20b-81d8-4716-994a-d8c87688c148 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:02.296183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d908b-b9bb-4b59-985e-3f0da312f470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:02.296965 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d908b-b9bb-4b59-985e-3f0da312f470 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:02.297247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d908b-b9bb-4b59-985e-3f0da312f470 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:02.297521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d908b-b9bb-4b59-985e-3f0da312f470 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:02.304030 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d908b-b9bb-4b59-985e-3f0da312f470 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:02.305511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d908b-b9bb-4b59-985e-3f0da312f470 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:02.320209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-0f2d908b-b9bb-4b59-985e-3f0da312f470 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:16:02.320209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:02.320209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:02.320209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:02.320209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:02.320209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:02.320209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:02.320209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:02.320209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:02.320209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:02.320209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:02.320209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:02.320209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:16:02.320209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:16:02.320209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:16:02.321335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:16:02.321335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:02.321335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:02.321335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:02.321335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:02.321335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:02.321335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:02.321335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:02.321335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:02.321335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:02.321335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:02.321335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:02.321335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:02.321335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:02.321335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:02.321335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:02.321335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:02.322337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:16:02.322337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:16:02.322337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:16:02.322337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:16:02.322337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:16:02.322337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:16:02.322337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:16:02.322337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:16:02.322337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 622/1258] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:16:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:16:02.327991 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3ef46-3366-4b98-8886-a96c89d7ab6b None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:02.426690 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3ef46-3366-4b98-8886-a96c89d7ab6b None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:02.459392 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b1bf2-bc22-4586-969a-f784602fada5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:02.460052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b1bf2-bc22-4586-969a-f784602fada5 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:02.460337 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b1bf2-bc22-4586-969a-f784602fada5 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:02.460616 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b1bf2-bc22-4586-969a-f784602fada5 None admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:02.461400 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b1bf2-bc22-4586-969a-f784602fada5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:02.461713 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b1bf2-bc22-4586-969a-f784602fada5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:02.463187 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b1bf2-bc22-4586-969a-f784602fada5 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:02.475646 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 637/1259] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:16:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:02.481687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d908b-b9bb-4b59-985e-3f0da312f470 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:02.509287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:02.580130 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d908b-b9bb-4b59-985e-3f0da312f470 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:02.614423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd9567-8d35-4aad-ba30-731d2534142e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:02.617957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd9567-8d35-4aad-ba30-731d2534142e None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:02.617957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd9567-8d35-4aad-ba30-731d2534142e None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:02.617957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd9567-8d35-4aad-ba30-731d2534142e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:02.617957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd9567-8d35-4aad-ba30-731d2534142e None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=2192547babde4449a46f9cb37ed64524) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:02.617957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd9567-8d35-4aad-ba30-731d2534142e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:02.618403 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd9567-8d35-4aad-ba30-731d2534142e None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:02.624442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 623/1260] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:16:02 2022] GET /identity/v3/role_assignments?user.id=2192547babde4449a46f9cb37ed64524&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=25516be1485c49538544ea8a596e7d17 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:04.655524 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3baa08-56f5-4903-929f-8d28dc5c3ae8 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:04.656082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3baa08-56f5-4903-929f-8d28dc5c3ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:04.656082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3baa08-56f5-4903-929f-8d28dc5c3ae8 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:04.657414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 638/1261] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:16:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:16:04.662955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23b445-914b-4fe6-85d8-3ff5ad8176e5 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:04.663377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23b445-914b-4fe6-85d8-3ff5ad8176e5 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:04.669422 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23b445-914b-4fe6-85d8-3ff5ad8176e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:04.680386 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-5a23b445-914b-4fe6-85d8-3ff5ad8176e5 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:16:04.681570 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5a23b445-914b-4fe6-85d8-3ff5ad8176e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:16:04.746838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 624/1262] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:16:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:16:04.769683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3baa08-56f5-4903-929f-8d28dc5c3ae8 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:04.780944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:04.868315 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3baa08-56f5-4903-929f-8d28dc5c3ae8 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:04.908534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2003ac8-3ddc-4c39-b932-a900ab5d2d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:04.909284 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2003ac8-3ddc-4c39-b932-a900ab5d2d92 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:04.909564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2003ac8-3ddc-4c39-b932-a900ab5d2d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:04.909835 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2003ac8-3ddc-4c39-b932-a900ab5d2d92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:04.920105 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2003ac8-3ddc-4c39-b932-a900ab5d2d92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:16:04.927717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-a2003ac8-3ddc-4c39-b932-a900ab5d2d92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:16:04.927717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:04.927717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:04.927717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:04.927717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:04.927717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:04.927717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:04.927717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:04.927717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:04.927717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:04.927717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:04.927717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:04.927717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:16:04.927717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:16:04.927717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:16:04.927717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:16:04.928953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:16:04.928953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:16:04.928953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:16:04.928953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:16:04.928953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:04.928953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:04.928953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:04.928953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:04.928953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:04.928953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:04.928953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:04.928953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:04.928953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:04.928953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:04.928953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:04.928953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:04.930048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:04.930048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:04.930048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:04.930048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:04.930048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:16:04.930048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:16:04.930048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:16:04.930048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:16:04.930048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:16:04.930048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:16:04.930048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:16:04.930048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:16:04.930048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 639/1263] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:16:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:16:04.936513 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23b445-914b-4fe6-85d8-3ff5ad8176e5 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:04.953725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:05.031946 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23b445-914b-4fe6-85d8-3ff5ad8176e5 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:05.065496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a854f-d7f9-4e6b-9a1f-f8b14da3be66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:05.066151 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a854f-d7f9-4e6b-9a1f-f8b14da3be66 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:05.066459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a854f-d7f9-4e6b-9a1f-f8b14da3be66 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:05.066734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a854f-d7f9-4e6b-9a1f-f8b14da3be66 None admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:05.067472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a854f-d7f9-4e6b-9a1f-f8b14da3be66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:05.067808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a854f-d7f9-4e6b-9a1f-f8b14da3be66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:05.069150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a854f-d7f9-4e6b-9a1f-f8b14da3be66 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:05.076950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 625/1264] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:16:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:05.082945 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2003ac8-3ddc-4c39-b932-a900ab5d2d92 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:05.152071 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:05.184973 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2003ac8-3ddc-4c39-b932-a900ab5d2d92 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:05.218474 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f974535-a6c4-4577-93a9-235f66b79b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:05.219262 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f974535-a6c4-4577-93a9-235f66b79b35 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:05.219543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f974535-a6c4-4577-93a9-235f66b79b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:05.219816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f974535-a6c4-4577-93a9-235f66b79b35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:05.230279 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f974535-a6c4-4577-93a9-235f66b79b35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:05.231747 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f974535-a6c4-4577-93a9-235f66b79b35 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:05.252660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-8f974535-a6c4-4577-93a9-235f66b79b35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:16:05.252660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:05.252660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:05.252660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:05.252660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:05.252660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:05.252660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:05.252660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:05.252660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:05.252660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:05.252660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:05.252660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:05.252660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:16:05.252660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:16:05.252660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:16:05.253764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:16:05.253764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:05.253764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:05.253764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:05.253764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:05.253764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:05.253764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:05.253764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:05.253764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:05.253764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:05.253764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:05.253764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:05.253764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:05.253764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:05.253764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:05.253764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:05.253764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:05.259031 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:16:05.259031 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:16:05.259031 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:16:05.259031 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:16:05.259031 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 640/1265] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:16:05.260716 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a854f-d7f9-4e6b-9a1f-f8b14da3be66 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:05.345664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:05.370359 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a854f-d7f9-4e6b-9a1f-f8b14da3be66 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:05.404484 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d0e59-b3b5-4a69-b245-f299f7f9acf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:05.405146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d0e59-b3b5-4a69-b245-f299f7f9acf3 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:05.405431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d0e59-b3b5-4a69-b245-f299f7f9acf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:05.405710 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d0e59-b3b5-4a69-b245-f299f7f9acf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:05.406466 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d0e59-b3b5-4a69-b245-f299f7f9acf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:05.406778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d0e59-b3b5-4a69-b245-f299f7f9acf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:05.408144 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d0e59-b3b5-4a69-b245-f299f7f9acf3 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:05.420867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 626/1266] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:05.426441 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f974535-a6c4-4577-93a9-235f66b79b35 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:05.531305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f974535-a6c4-4577-93a9-235f66b79b35 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:05.550471 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:05.563807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0556de9-1e0c-4d86-a13b-dfd9b860bcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:05.564624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0556de9-1e0c-4d86-a13b-dfd9b860bcb5 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:05.564910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0556de9-1e0c-4d86-a13b-dfd9b860bcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:05.565191 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0556de9-1e0c-4d86-a13b-dfd9b860bcb5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:05.572837 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0556de9-1e0c-4d86-a13b-dfd9b860bcb5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:05.574391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0556de9-1e0c-4d86-a13b-dfd9b860bcb5 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:05.589029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-f0556de9-1e0c-4d86-a13b-dfd9b860bcb5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:16:05.589029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:05.589029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:05.589029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:05.589029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:05.589029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:05.589029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:05.589029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:05.589029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:05.589029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:05.589029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:05.589029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:05.589029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:16:05.589029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:16:05.589029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:16:05.589029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:05.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:05.592119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:05.592119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:05.592119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:16:05.592119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:16:05.592119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:16:05.592119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:16:05.592119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:16:05.592119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:16:05.592119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:16:05.592119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:16:05.592119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 641/1267] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:16:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:16:05.596713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d0e59-b3b5-4a69-b245-f299f7f9acf3 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:05.695711 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d0e59-b3b5-4a69-b245-f299f7f9acf3 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:05.732814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a16a06-7ea0-4933-9736-87db72594a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:05.733444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a16a06-7ea0-4933-9736-87db72594a83 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:05.733444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a16a06-7ea0-4933-9736-87db72594a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:05.733650 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a16a06-7ea0-4933-9736-87db72594a83 None admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:05.734575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a16a06-7ea0-4933-9736-87db72594a83 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:05.734887 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a16a06-7ea0-4933-9736-87db72594a83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:05.736384 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a16a06-7ea0-4933-9736-87db72594a83 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:05.746403 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:05.754525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 627/1268] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Jul 20 15:16:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:05.760184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0556de9-1e0c-4d86-a13b-dfd9b860bcb5 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:05.859411 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0556de9-1e0c-4d86-a13b-dfd9b860bcb5 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:05.892530 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c9b48-98d5-43e0-bc1e-d5b4c2e4e89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:05.893315 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c9b48-98d5-43e0-bc1e-d5b4c2e4e89f None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:05.893592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c9b48-98d5-43e0-bc1e-d5b4c2e4e89f None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:05.893862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c9b48-98d5-43e0-bc1e-d5b4c2e4e89f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:05.904050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c9b48-98d5-43e0-bc1e-d5b4c2e4e89f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:05.905468 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c9b48-98d5-43e0-bc1e-d5b4c2e4e89f None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:05.922203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-311c9b48-98d5-43e0-bc1e-d5b4c2e4e89f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:16:05.922203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:05.922203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:05.922203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:05.922203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:05.922203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:05.922203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:05.922203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:05.922203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:05.922203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:05.922203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:05.922203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:05.922203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:16:05.922203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:16:05.922203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:16:05.925270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:16:05.925270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:05.925270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:05.925270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:05.925270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:05.925270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:05.925270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:05.925270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:05.925270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:05.925270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:05.925270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:05.925270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:05.925270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:05.925270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:05.925270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:05.925270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:05.925270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:05.926430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:16:05.926430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:16:05.926430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:16:05.926430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:16:05.926430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 642/1269] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:16:05.929254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a16a06-7ea0-4933-9736-87db72594a83 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:06.031337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a16a06-7ea0-4933-9736-87db72594a83 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:06.065719 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf62fc3-2be5-4d1e-8a49-7e36a040c3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:06.066509 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf62fc3-2be5-4d1e-8a49-7e36a040c3ce None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:06.066799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf62fc3-2be5-4d1e-8a49-7e36a040c3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:06.067074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf62fc3-2be5-4d1e-8a49-7e36a040c3ce None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:06.067803 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf62fc3-2be5-4d1e-8a49-7e36a040c3ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:06.068112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf62fc3-2be5-4d1e-8a49-7e36a040c3ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:06.069546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf62fc3-2be5-4d1e-8a49-7e36a040c3ce None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:06.081541 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 628/1270] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:06.087310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c9b48-98d5-43e0-bc1e-d5b4c2e4e89f None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:06.119734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:06.187162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c9b48-98d5-43e0-bc1e-d5b4c2e4e89f None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:06.219223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10664554-e905-4c67-81eb-ff8025e2be85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:06.220006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10664554-e905-4c67-81eb-ff8025e2be85 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:06.220287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10664554-e905-4c67-81eb-ff8025e2be85 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:06.220555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10664554-e905-4c67-81eb-ff8025e2be85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:06.227078 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10664554-e905-4c67-81eb-ff8025e2be85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:06.228553 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10664554-e905-4c67-81eb-ff8025e2be85 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:06.242498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-10664554-e905-4c67-81eb-ff8025e2be85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:16:06.242498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:06.242498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:06.242498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:06.242498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:06.242498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:06.242498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:06.242498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:06.242498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:06.242498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:06.242498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:06.242498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:06.242498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:16:06.242498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:16:06.242498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:16:06.243589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:16:06.243589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:06.243589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:06.243589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:06.243589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:06.243589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:06.243589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:06.243589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:06.243589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:06.243589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:06.243589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:06.243589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:06.243589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:06.243589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:06.243589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:06.243589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:06.243589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:06.244709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:16:06.244709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:16:06.244709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:16:06.244709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:16:06.244709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:16:06.244709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:16:06.244709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:16:06.244709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:16:06.244709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 643/1271] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:16:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:16:06.249867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf62fc3-2be5-4d1e-8a49-7e36a040c3ce None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:06.307732 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:06.348087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf62fc3-2be5-4d1e-8a49-7e36a040c3ce None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:06.380581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc3d40-2b71-47f6-9cca-167ef698c0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:06.381195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc3d40-2b71-47f6-9cca-167ef698c0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:06.381575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc3d40-2b71-47f6-9cca-167ef698c0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:06.381855 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc3d40-2b71-47f6-9cca-167ef698c0c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:06.382678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc3d40-2b71-47f6-9cca-167ef698c0c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:06.382993 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc3d40-2b71-47f6-9cca-167ef698c0c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:06.384413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc3d40-2b71-47f6-9cca-167ef698c0c7 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:06.396999 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 629/1272] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:16:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:06.403391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10664554-e905-4c67-81eb-ff8025e2be85 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:06.494210 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:06.502574 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10664554-e905-4c67-81eb-ff8025e2be85 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:06.534923 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d8b92-6f70-4d70-8818-cb2a2e232ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:06.535804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d8b92-6f70-4d70-8818-cb2a2e232ab9 None admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:06.536084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d8b92-6f70-4d70-8818-cb2a2e232ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:06.536380 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d8b92-6f70-4d70-8818-cb2a2e232ab9 None admin] PATH_INFO: `/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/2192547babde4449a46f9cb37ed64524/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:06.538301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d8b92-6f70-4d70-8818-cb2a2e232ab9 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdae1cb71dd449b8a18062b7b52889e4, user_id=2192547babde4449a46f9cb37ed64524, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:06.539726 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d8b92-6f70-4d70-8818-cb2a2e232ab9 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:06.628094 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 644/1273] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Jul 20 15:16:06 2022] PUT /identity/v3/projects/cdae1cb71dd449b8a18062b7b52889e4/users/2192547babde4449a46f9cb37ed64524/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:16:08.656214 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6bd64-3376-406e-9661-3a7905b95cbe None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:08.656743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6bd64-3376-406e-9661-3a7905b95cbe None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:08.656743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6bd64-3376-406e-9661-3a7905b95cbe None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:08.658130 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 630/1274] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:16:08.663390 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44718b6-097d-4179-af4a-3f9501cdf9c4 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:08.667801 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44718b6-097d-4179-af4a-3f9501cdf9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:08.667801 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44718b6-097d-4179-af4a-3f9501cdf9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:08.680964 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-e44718b6-097d-4179-af4a-3f9501cdf9c4 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:16:08.682154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e44718b6-097d-4179-af4a-3f9501cdf9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:16:08.747549 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 645/1275] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:16:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:16:08.771136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6bd64-3376-406e-9661-3a7905b95cbe None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:08.871825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6bd64-3376-406e-9661-3a7905b95cbe None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:08.888764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:08.905359 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804d65b-e9e6-4490-b224-d7cf1f3de847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:08.906159 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804d65b-e9e6-4490-b224-d7cf1f3de847 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:08.906525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804d65b-e9e6-4490-b224-d7cf1f3de847 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:08.906753 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804d65b-e9e6-4490-b224-d7cf1f3de847 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:08.916682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804d65b-e9e6-4490-b224-d7cf1f3de847 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 15:16:08.924918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-3804d65b-e9e6-4490-b224-d7cf1f3de847 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:16:08.924918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:08.924918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:08.924918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:08.924918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:08.924918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:08.924918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:08.924918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:08.924918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:08.924918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:08.924918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:08.924918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:08.924918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 15:16:08.924918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 15:16:08.924918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 15:16:08.924918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 15:16:08.926087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 15:16:08.926087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 15:16:08.926087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 15:16:08.926087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 15:16:08.926087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:08.926087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:08.926087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:08.926087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:08.926087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:08.926087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:08.926087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:08.926087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:08.926087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:08.926087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:08.926087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:08.926087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:08.927173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:08.927173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:08.927173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:08.927173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:08.927173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 15:16:08.927173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 15:16:08.927173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 15:16:08.927173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 15:16:08.927173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 15:16:08.927173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 15:16:08.927173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 15:16:08.927173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:16:08.927173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 631/1276] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Jul 20 15:16:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:16:08.932183 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44718b6-097d-4179-af4a-3f9501cdf9c4 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:09.012116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:09.029275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44718b6-097d-4179-af4a-3f9501cdf9c4 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:09.062692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792cdee-086f-4846-a150-46751c6024a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:09.063354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792cdee-086f-4846-a150-46751c6024a3 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:09.063637 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792cdee-086f-4846-a150-46751c6024a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:09.063913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792cdee-086f-4846-a150-46751c6024a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:09.064649 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792cdee-086f-4846-a150-46751c6024a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:09.064955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792cdee-086f-4846-a150-46751c6024a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:09.066348 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792cdee-086f-4846-a150-46751c6024a3 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:09.074089 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 646/1277] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Jul 20 15:16:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:09.079923 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804d65b-e9e6-4490-b224-d7cf1f3de847 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:09.179018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804d65b-e9e6-4490-b224-d7cf1f3de847 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:09.211289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1edb5-76ef-47d5-9f50-f5535a797681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:09.212076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1edb5-76ef-47d5-9f50-f5535a797681 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:09.212359 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1edb5-76ef-47d5-9f50-f5535a797681 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:09.212635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1edb5-76ef-47d5-9f50-f5535a797681 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:09.223514 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1edb5-76ef-47d5-9f50-f5535a797681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:09.224968 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1edb5-76ef-47d5-9f50-f5535a797681 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:09.243152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-bcd1edb5-76ef-47d5-9f50-f5535a797681 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:16:09.243152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:09.243152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:09.243152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:09.243152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:09.243152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:09.243152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:09.243152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:09.243152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:09.243152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:09.243152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:09.243152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:09.243152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:16:09.243152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:16:09.243152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:16:09.244247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:16:09.244247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:09.244247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:09.244247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:09.244247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:09.244247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:09.244247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:09.244247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:09.244247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:09.244247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:09.244247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:09.244247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:09.244247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:09.244247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:09.244247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:09.244247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:09.244247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:09.245389 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:16:09.245389 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:16:09.245389 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:16:09.245389 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:16:09.245389 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 632/1278] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:16:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:16:09.250694 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792cdee-086f-4846-a150-46751c6024a3 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:09.352108 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792cdee-086f-4846-a150-46751c6024a3 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:09.384790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f2632-d9e3-49f9-97bc-11b3df12d1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:09.385493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f2632-d9e3-49f9-97bc-11b3df12d1ba None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:09.385787 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f2632-d9e3-49f9-97bc-11b3df12d1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:09.386053 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f2632-d9e3-49f9-97bc-11b3df12d1ba None admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:09.386793 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f2632-d9e3-49f9-97bc-11b3df12d1ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:09.387105 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f2632-d9e3-49f9-97bc-11b3df12d1ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:09.388472 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f2632-d9e3-49f9-97bc-11b3df12d1ba None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:09.400982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 647/1279] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:09.407394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1edb5-76ef-47d5-9f50-f5535a797681 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:09.440291 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:09.507804 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1edb5-76ef-47d5-9f50-f5535a797681 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:09.540941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a586d5-84d6-4978-9f6d-b2429f51f18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:09.541727 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a586d5-84d6-4978-9f6d-b2429f51f18c None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:09.542011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a586d5-84d6-4978-9f6d-b2429f51f18c None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:09.542309 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a586d5-84d6-4978-9f6d-b2429f51f18c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:09.549780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a586d5-84d6-4978-9f6d-b2429f51f18c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:09.551237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a586d5-84d6-4978-9f6d-b2429f51f18c None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:09.565527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-63a586d5-84d6-4978-9f6d-b2429f51f18c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:16:09.565527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:09.565527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:09.565527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:09.565527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:09.565527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:09.565527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:09.565527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:09.565527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:09.565527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:09.565527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:09.565527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:09.565527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 15:16:09.565527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 15:16:09.565527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 15:16:09.565527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:09.566721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:09.568045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:09.568045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:09.568045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 15:16:09.568045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 15:16:09.568045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 15:16:09.568045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 15:16:09.568045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 15:16:09.568045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 15:16:09.568045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 15:16:09.568045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:16:09.568045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 633/1280] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:16:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 15:16:09.572565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f2632-d9e3-49f9-97bc-11b3df12d1ba None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:09.576379 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:09.669019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f2632-d9e3-49f9-97bc-11b3df12d1ba None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:09.702343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c1f45-3c3c-4f76-9311-0a4124184ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:09.702960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c1f45-3c3c-4f76-9311-0a4124184ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:09.703242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c1f45-3c3c-4f76-9311-0a4124184ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:09.703515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c1f45-3c3c-4f76-9311-0a4124184ec0 None admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:09.704331 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c1f45-3c3c-4f76-9311-0a4124184ec0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:09.704636 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c1f45-3c3c-4f76-9311-0a4124184ec0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:09.706175 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c1f45-3c3c-4f76-9311-0a4124184ec0 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:09.723300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 648/1281] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Jul 20 15:16:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:09.729082 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a586d5-84d6-4978-9f6d-b2429f51f18c None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:09.817665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:09.826906 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a586d5-84d6-4978-9f6d-b2429f51f18c None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:09.860237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97753698-38e1-43df-b47b-6fd2142cdd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:09.860878 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97753698-38e1-43df-b47b-6fd2142cdd89 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:09.861161 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97753698-38e1-43df-b47b-6fd2142cdd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:09.861438 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97753698-38e1-43df-b47b-6fd2142cdd89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:09.926650 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97753698-38e1-43df-b47b-6fd2142cdd89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:09.928100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97753698-38e1-43df-b47b-6fd2142cdd89 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:10.014715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:10.024443 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 634/1282] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Jul 20 15:16:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 15:16:10.030963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c1f45-3c3c-4f76-9311-0a4124184ec0 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:10.092945 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:10.129304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c1f45-3c3c-4f76-9311-0a4124184ec0 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:10.163538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04be52a-1382-4ed4-bb8f-6e6c5ad2adff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:10.164332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04be52a-1382-4ed4-bb8f-6e6c5ad2adff None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:10.164616 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04be52a-1382-4ed4-bb8f-6e6c5ad2adff None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:10.164930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04be52a-1382-4ed4-bb8f-6e6c5ad2adff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:10.175409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04be52a-1382-4ed4-bb8f-6e6c5ad2adff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:10.176846 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04be52a-1382-4ed4-bb8f-6e6c5ad2adff None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:10.193632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-e04be52a-1382-4ed4-bb8f-6e6c5ad2adff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:16:10.193632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:10.193632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:10.193632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:10.193632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:10.193632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:10.193632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:10.193632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:10.193632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:10.193632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:10.193632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:10.193632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:10.193632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 15:16:10.193632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 15:16:10.193632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 15:16:10.194777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 15:16:10.194777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:10.194777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:10.194777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:10.194777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:10.194777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:10.194777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:10.194777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:10.194777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:10.194777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:10.194777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:10.194777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:10.194777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:10.194777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:10.194777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:10.194777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:10.194777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:10.196114 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 15:16:10.196114 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 15:16:10.196114 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 15:16:10.196114 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:16:10.196114 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 649/1283] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:16:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:16:10.200736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97753698-38e1-43df-b47b-6fd2142cdd89 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:10.299697 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97753698-38e1-43df-b47b-6fd2142cdd89 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:10.334079 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70247e-34e5-44b9-b1b1-3f7922f247ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:10.334798 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70247e-34e5-44b9-b1b1-3f7922f247ed None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:10.335083 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70247e-34e5-44b9-b1b1-3f7922f247ed None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:10.335360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70247e-34e5-44b9-b1b1-3f7922f247ed None admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:10.336073 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70247e-34e5-44b9-b1b1-3f7922f247ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:10.336379 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70247e-34e5-44b9-b1b1-3f7922f247ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:10.337730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70247e-34e5-44b9-b1b1-3f7922f247ed None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:10.348800 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 635/1284] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Jul 20 15:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:10.354465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04be52a-1382-4ed4-bb8f-6e6c5ad2adff None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:10.451038 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04be52a-1382-4ed4-bb8f-6e6c5ad2adff None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:10.464755 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:10.484437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d4a3b-5c8b-4e04-8221-7316968186c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:10.485229 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d4a3b-5c8b-4e04-8221-7316968186c7 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:10.485511 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d4a3b-5c8b-4e04-8221-7316968186c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:10.485785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d4a3b-5c8b-4e04-8221-7316968186c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:10.492568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d4a3b-5c8b-4e04-8221-7316968186c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:10.494001 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d4a3b-5c8b-4e04-8221-7316968186c7 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:10.508913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application [None req-874d4a3b-5c8b-4e04-8221-7316968186c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:16:10.508913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:10.508913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:10.508913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:10.508913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:10.508913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:10.508913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:10.508913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:10.508913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:10.508913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:10.508913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:10.508913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:10.508913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 15:16:10.508913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 15:16:10.508913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 15:16:10.510003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 15:16:10.510003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:10.510003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:10.510003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:10.510003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:10.510003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:10.510003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:10.510003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:10.510003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:10.510003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:10.510003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:10.510003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:10.510003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:10.510003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:10.510003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:10.510003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:10.510003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:10.511340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 15:16:10.511340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 15:16:10.511340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 15:16:10.511340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 15:16:10.511340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 15:16:10.511340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 15:16:10.511340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 15:16:10.511340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: ERROR keystone.server.flask.application Jul 20 15:16:10.511340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 650/1285] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Jul 20 15:16:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:16:10.516570 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70247e-34e5-44b9-b1b1-3f7922f247ed None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:10.565048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:10.613439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70247e-34e5-44b9-b1b1-3f7922f247ed None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:10.653992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d321a-b3a6-4790-8e7d-0944e5ddcab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:10.654678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d321a-b3a6-4790-8e7d-0944e5ddcab5 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:10.654958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d321a-b3a6-4790-8e7d-0944e5ddcab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:10.655231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d321a-b3a6-4790-8e7d-0944e5ddcab5 None admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:10.656016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d321a-b3a6-4790-8e7d-0944e5ddcab5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:10.656322 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d321a-b3a6-4790-8e7d-0944e5ddcab5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:10.657737 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d321a-b3a6-4790-8e7d-0944e5ddcab5 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:10.670722 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 636/1286] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Jul 20 15:16:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:10.677303 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d4a3b-5c8b-4e04-8221-7316968186c7 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:10.776000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d4a3b-5c8b-4e04-8221-7316968186c7 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:10.808564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d675dcde-4131-486d-bdac-53628759bf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:10.809204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675dcde-4131-486d-bdac-53628759bf20 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:10.809484 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675dcde-4131-486d-bdac-53628759bf20 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:10.809757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675dcde-4131-486d-bdac-53628759bf20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:10.810605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d675dcde-4131-486d-bdac-53628759bf20 None admin] RBAC: Adding query filter params (role.id=25516be1485c49538544ea8a596e7d17, scope.project.id=cdae1cb71dd449b8a18062b7b52889e4, user.id=2192547babde4449a46f9cb37ed64524) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:10.811502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d675dcde-4131-486d-bdac-53628759bf20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:10.812919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d675dcde-4131-486d-bdac-53628759bf20 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:10.824563 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 651/1287] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Jul 20 15:16:10 2022] GET /identity/v3/role_assignments?user.id=2192547babde4449a46f9cb37ed64524&scope.project.id=cdae1cb71dd449b8a18062b7b52889e4&role.id=25516be1485c49538544ea8a596e7d17 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:12.897770 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9519aa0-766c-49d2-aa68-0d70db33dd52 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:12.898387 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9519aa0-766c-49d2-aa68-0d70db33dd52 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:12.898387 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9519aa0-766c-49d2-aa68-0d70db33dd52 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:12.899930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 637/1288] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:16:12.905141 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d1981-f2cc-47f2-acf9-b4688fab84ff None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:12.905542 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d1981-f2cc-47f2-acf9-b4688fab84ff None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:12.905704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d1981-f2cc-47f2-acf9-b4688fab84ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:12.923561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-745d1981-f2cc-47f2-acf9-b4688fab84ff None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:16:12.925877 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-745d1981-f2cc-47f2-acf9-b4688fab84ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:16:12.954743 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:12.991887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 652/1289] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:16:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:16:13.015434 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9519aa0-766c-49d2-aa68-0d70db33dd52 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:13.114247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9519aa0-766c-49d2-aa68-0d70db33dd52 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:13.125763 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:13.148642 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e2bd5-2bc6-4138-a8b6-774317c9c8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:13.149382 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e2bd5-2bc6-4138-a8b6-774317c9c8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:13.149652 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e2bd5-2bc6-4138-a8b6-774317c9c8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:13.149926 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e2bd5-2bc6-4138-a8b6-774317c9c8f7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:13.150613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e2bd5-2bc6-4138-a8b6-774317c9c8f7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:13.152001 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e2bd5-2bc6-4138-a8b6-774317c9c8f7 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:13.163797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-7d9e2bd5-2bc6-4138-a8b6-774317c9c8f7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:16:13.163797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:13.163797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:16:13.163797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:16:13.163797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:16:13.163797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:16:13.163797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:16:13.163797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:16:13.163797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:16:13.163797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:16:13.163797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:16:13.163797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:16:13.163797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:13.163797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:13.163797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:13.163797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:13.163797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:13.169087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:13.169087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:13.169087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:13.169087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:13.169087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:13.169087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:13.169087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:16:13.169087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:16:13.169087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:16:13.169087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:16:13.169087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:13.169087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:13.169087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:13.169087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:13.169087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:13.169087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:13.170129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:13.170129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:13.170129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:13.170129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:13.170129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:13.170129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:13.170129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:13.170129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:13.170129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:13.170129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:13.170129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:16:13.170129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:16:13.170129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:16:13.170129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:16:13.170129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 638/1290] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Jul 20 15:16:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:16:13.171623 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d1981-f2cc-47f2-acf9-b4688fab84ff None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:13.267219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d1981-f2cc-47f2-acf9-b4688fab84ff None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:13.300598 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c268e4-d78b-4869-b9dc-4d0546ba499b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:13.301285 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c268e4-d78b-4869-b9dc-4d0546ba499b None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:13.301569 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c268e4-d78b-4869-b9dc-4d0546ba499b None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:13.301846 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c268e4-d78b-4869-b9dc-4d0546ba499b None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:13.302603 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c268e4-d78b-4869-b9dc-4d0546ba499b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:13.302912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c268e4-d78b-4869-b9dc-4d0546ba499b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:13.304356 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c268e4-d78b-4869-b9dc-4d0546ba499b None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:13.310034 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 653/1291] 10.222.0.78 () {56 vars in 1116 bytes} [Wed Jul 20 15:16:13 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:13.316291 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e2bd5-2bc6-4138-a8b6-774317c9c8f7 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:13.426309 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e2bd5-2bc6-4138-a8b6-774317c9c8f7 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:13.468476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d561aa-0033-4cf3-988b-79eaf7c6f617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:13.469121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d561aa-0033-4cf3-988b-79eaf7c6f617 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:13.469410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d561aa-0033-4cf3-988b-79eaf7c6f617 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:13.469678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d561aa-0033-4cf3-988b-79eaf7c6f617 None admin] PATH_INFO: `/v3/services` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:13.470425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d561aa-0033-4cf3-988b-79eaf7c6f617 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:13.470735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d561aa-0033-4cf3-988b-79eaf7c6f617 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:13.472108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d561aa-0033-4cf3-988b-79eaf7c6f617 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:13.476992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 639/1292] 10.222.0.78 () {56 vars in 1116 bytes} [Wed Jul 20 15:16:13 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:15.505262 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd75085-aa94-4c8b-985c-9060cff95316 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:15.505772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd75085-aa94-4c8b-985c-9060cff95316 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:15.505772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd75085-aa94-4c8b-985c-9060cff95316 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:15.507102 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 654/1293] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:16:15.512358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104fc68-5262-4bcb-beb5-844340cb67d6 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:15.512771 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104fc68-5262-4bcb-beb5-844340cb67d6 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:15.512938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104fc68-5262-4bcb-beb5-844340cb67d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:15.529946 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-d104fc68-5262-4bcb-beb5-844340cb67d6 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:16:15.531179 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d104fc68-5262-4bcb-beb5-844340cb67d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:16:15.559075 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:15.594604 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 640/1294] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:16:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:16:15.616946 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd75085-aa94-4c8b-985c-9060cff95316 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:15.648698 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:15.717300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd75085-aa94-4c8b-985c-9060cff95316 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:15.750883 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517152cf-e817-44d8-a4fb-f19c099a9480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:15.751548 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517152cf-e817-44d8-a4fb-f19c099a9480 None admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:15.751844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517152cf-e817-44d8-a4fb-f19c099a9480 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:15.752143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517152cf-e817-44d8-a4fb-f19c099a9480 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:15.752842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517152cf-e817-44d8-a4fb-f19c099a9480 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:15.754207 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517152cf-e817-44d8-a4fb-f19c099a9480 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:15.833017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 655/1295] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Jul 20 15:16:15 2022] POST /identity/v3/services => generated 246 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:16:17.888584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef13cf6-f439-44db-91d1-97dafef456d4 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:17.889151 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef13cf6-f439-44db-91d1-97dafef456d4 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:17.889151 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef13cf6-f439-44db-91d1-97dafef456d4 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:17.890577 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 641/1296] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:16:17.895606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15f7a0-6d1d-4c5e-8cfd-5786420123ae None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:17.896027 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15f7a0-6d1d-4c5e-8cfd-5786420123ae None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:17.896145 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15f7a0-6d1d-4c5e-8cfd-5786420123ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:17.913091 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-ac15f7a0-6d1d-4c5e-8cfd-5786420123ae None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:16:17.914292 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ac15f7a0-6d1d-4c5e-8cfd-5786420123ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:16:17.988380 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 656/1297] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:16:17 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:16:18.011304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef13cf6-f439-44db-91d1-97dafef456d4 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:18.118650 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef13cf6-f439-44db-91d1-97dafef456d4 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:18.152396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db8563-d5ab-43cc-9f39-fdf697001440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:18.153124 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db8563-d5ab-43cc-9f39-fdf697001440 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:18.153407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db8563-d5ab-43cc-9f39-fdf697001440 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:18.153684 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db8563-d5ab-43cc-9f39-fdf697001440 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:18.154363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db8563-d5ab-43cc-9f39-fdf697001440 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:18.155715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db8563-d5ab-43cc-9f39-fdf697001440 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:18.167520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-96db8563-d5ab-43cc-9f39-fdf697001440 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:16:18.167520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:18.167520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:16:18.167520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:16:18.167520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:16:18.167520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:16:18.167520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:16:18.167520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:16:18.167520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:16:18.167520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:16:18.167520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:16:18.167520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:16:18.167520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:18.167520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:18.167520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:18.167520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:18.167520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:18.168728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:18.168728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:18.168728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:18.168728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:18.168728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:18.168728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:18.168728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:16:18.168728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:16:18.168728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:16:18.168728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:16:18.168728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:18.168728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:18.168728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:18.168728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:18.168728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:18.168728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:18.169622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:18.169622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:18.169622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:18.169622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:18.169622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:18.169622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:18.169622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:18.169622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:18.169622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:18.169622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:18.169622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:16:18.169622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:16:18.169622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:16:18.169622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:16:18.169622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 642/1298] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Jul 20 15:16:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:16:18.175263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15f7a0-6d1d-4c5e-8cfd-5786420123ae None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:18.193396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:18.269854 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15f7a0-6d1d-4c5e-8cfd-5786420123ae None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:18.303061 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c978f5-367c-46eb-9ca5-85fa16034256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:18.303718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c978f5-367c-46eb-9ca5-85fa16034256 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:18.304003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c978f5-367c-46eb-9ca5-85fa16034256 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:18.304281 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c978f5-367c-46eb-9ca5-85fa16034256 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:18.304978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c978f5-367c-46eb-9ca5-85fa16034256 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:18.305370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c978f5-367c-46eb-9ca5-85fa16034256 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:18.306770 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c978f5-367c-46eb-9ca5-85fa16034256 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:18.312388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 657/1299] 10.222.0.78 () {56 vars in 1116 bytes} [Wed Jul 20 15:16:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:18.318481 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db8563-d5ab-43cc-9f39-fdf697001440 None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:18.348697 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:18.412948 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db8563-d5ab-43cc-9f39-fdf697001440 None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:18.445231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75549520-07ae-422f-b5b6-084c18949182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:18.445870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75549520-07ae-422f-b5b6-084c18949182 None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:18.446149 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75549520-07ae-422f-b5b6-084c18949182 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:18.446449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75549520-07ae-422f-b5b6-084c18949182 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:18.447228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75549520-07ae-422f-b5b6-084c18949182 None admin] RBAC: Adding query filter params (interface=public, service_id=4920d70ff2974f28bf03e6fa139a915e, region_id=RegionOne) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:18.447540 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75549520-07ae-422f-b5b6-084c18949182 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:18.448891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75549520-07ae-422f-b5b6-084c18949182 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:18.453697 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 643/1300] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Jul 20 15:16:18 2022] GET /identity/v3/endpoints?service_id=4920d70ff2974f28bf03e6fa139a915e&interface=public®ion_id=RegionOne => generated 194 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:18.459316 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c978f5-367c-46eb-9ca5-85fa16034256 None admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:18.540788 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:18.558850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c978f5-367c-46eb-9ca5-85fa16034256 None admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:18.591461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcdcf2-cd0d-4862-964c-230478f54716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:18.592121 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcdcf2-cd0d-4862-964c-230478f54716 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:18.592406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcdcf2-cd0d-4862-964c-230478f54716 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:18.592685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcdcf2-cd0d-4862-964c-230478f54716 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:18.593876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcdcf2-cd0d-4862-964c-230478f54716 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:18.595293 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcdcf2-cd0d-4862-964c-230478f54716 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:18.601876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 658/1301] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Jul 20 15:16:18 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:16:20.649190 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7e956-3845-4650-82bb-7a7f97ed3014 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:20.649811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7e956-3845-4650-82bb-7a7f97ed3014 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:20.649811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7e956-3845-4650-82bb-7a7f97ed3014 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:20.651202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 644/1302] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:16:20.656270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9ecad-55a7-41ef-a141-454961760eec None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:20.660052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9ecad-55a7-41ef-a141-454961760eec None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:20.660052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9ecad-55a7-41ef-a141-454961760eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:20.675093 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-43b9ecad-55a7-41ef-a141-454961760eec None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:16:20.676330 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-43b9ecad-55a7-41ef-a141-454961760eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:16:20.739987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 659/1303] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:16:20 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 15:16:20.764439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7e956-3845-4650-82bb-7a7f97ed3014 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:20.869748 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:20.876888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7e956-3845-4650-82bb-7a7f97ed3014 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:20.910427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fcd5b9-eac3-4e38-8414-bf2076ed23ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:20.911207 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcd5b9-eac3-4e38-8414-bf2076ed23ae None admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:20.911485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcd5b9-eac3-4e38-8414-bf2076ed23ae None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:20.911759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcd5b9-eac3-4e38-8414-bf2076ed23ae None admin] PATH_INFO: `/v3/services/placement` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:20.912416 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fcd5b9-eac3-4e38-8414-bf2076ed23ae None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:20.913777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fcd5b9-eac3-4e38-8414-bf2076ed23ae None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:20.926647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application [None req-45fcd5b9-eac3-4e38-8414-bf2076ed23ae None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:16:20.926647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:20.926647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 15:16:20.926647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 15:16:20.926647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 15:16:20.926647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 15:16:20.926647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 15:16:20.926647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:16:20.926647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:16:20.926647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:16:20.926647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 15:16:20.926647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:16:20.926647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 15:16:20.926647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 15:16:20.926647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 15:16:20.926647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 15:16:20.926647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 15:16:20.928071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 15:16:20.928071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 15:16:20.928071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 15:16:20.928071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 15:16:20.928071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 15:16:20.928071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 15:16:20.928071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 15:16:20.928071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 15:16:20.928071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 15:16:20.928071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 15:16:20.928071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 15:16:20.928071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 15:16:20.928071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 15:16:20.928071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 15:16:20.928071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 15:16:20.928071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 15:16:20.929231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 15:16:20.929231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application with Lock( Jul 20 15:16:20.929231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 15:16:20.929231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self._enter() Jul 20 15:16:20.929231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 15:16:20.929231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 15:16:20.929231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 15:16:20.929231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application return self.creator() Jul 20 15:16:20.929231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 15:16:20.929231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application created_value = creator( Jul 20 15:16:20.929231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 15:16:20.929231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 15:16:20.929231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 15:16:20.929231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: ERROR keystone.server.flask.application Jul 20 15:16:20.929231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 645/1304] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Jul 20 15:16:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 15:16:20.934273 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9ecad-55a7-41ef-a141-454961760eec None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:21.009463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:21.031806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9ecad-55a7-41ef-a141-454961760eec None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:21.064104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db0f9e-8273-4e72-b290-aee827f6b4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:21.064757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db0f9e-8273-4e72-b290-aee827f6b4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:21.065069 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db0f9e-8273-4e72-b290-aee827f6b4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:21.065428 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db0f9e-8273-4e72-b290-aee827f6b4c5 None admin] PATH_INFO: `/v3/services` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:21.066126 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db0f9e-8273-4e72-b290-aee827f6b4c5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:16:21.066458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db0f9e-8273-4e72-b290-aee827f6b4c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:21.067829 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db0f9e-8273-4e72-b290-aee827f6b4c5 None admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:21.072849 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 660/1305] 10.222.0.78 () {56 vars in 1116 bytes} [Wed Jul 20 15:16:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:16:21.079629 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fcd5b9-eac3-4e38-8414-bf2076ed23ae None admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:16:21.186520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fcd5b9-eac3-4e38-8414-bf2076ed23ae None admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:16:21.219889 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f893b-1695-4276-994b-ea4758e6ac82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:16:21.226749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f893b-1695-4276-994b-ea4758e6ac82 None admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:16:21.226749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f893b-1695-4276-994b-ea4758e6ac82 None admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:16:21.226749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f893b-1695-4276-994b-ea4758e6ac82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:16:21.226749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f893b-1695-4276-994b-ea4758e6ac82 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:16:21.226749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f893b-1695-4276-994b-ea4758e6ac82 None admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:16:21.333948 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 646/1306] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Jul 20 15:16:21 2022] POST /identity/v3/endpoints => generated 331 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:16:31.174401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=30 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:31.329459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:53.029428 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238e23f-1407-41d8-b033-f3f747e75b14 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:53.030518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238e23f-1407-41d8-b033-f3f747e75b14 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:53.031019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238e23f-1407-41d8-b033-f3f747e75b14 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:53.038086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b06326-fd26-4cee-ad54-232ed4377586 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:53.042776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b06326-fd26-4cee-ad54-232ed4377586 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:53.042776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b06326-fd26-4cee-ad54-232ed4377586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:53.043642 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 661/1307] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:22:53 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:53.151172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-57b06326-fd26-4cee-ad54-232ed4377586 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:53.152718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-57b06326-fd26-4cee-ad54-232ed4377586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:53.310751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 647/1308] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:22:53.355661 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131a1e3-144d-4bf7-8a4e-51c5b3161071 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:53.355959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131a1e3-144d-4bf7-8a4e-51c5b3161071 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:53.356329 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131a1e3-144d-4bf7-8a4e-51c5b3161071 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:53.357733 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 662/1309] 10.222.0.78 () {54 vars in 965 bytes} [Wed Jul 20 15:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:53.369335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16961247-1314-423f-a8ec-807b780b3766 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:53.369703 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16961247-1314-423f-a8ec-807b780b3766 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:53.370061 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16961247-1314-423f-a8ec-807b780b3766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:53.480731 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-16961247-1314-423f-a8ec-807b780b3766 None None] MFA Rules not processed for user `b57530f18d9c4d328f8014db736d8de9`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:53.483382 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-16961247-1314-423f-a8ec-807b780b3766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:53.504607 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:53.565206 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 648/1310] 10.222.0.78 () {58 vars in 1062 bytes} [Wed Jul 20 15:22:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:22:53.575180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131a1e3-144d-4bf7-8a4e-51c5b3161071 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:53.577214 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-7131a1e3-144d-4bf7-8a4e-51c5b3161071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:53.694764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131a1e3-144d-4bf7-8a4e-51c5b3161071 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:53.773245 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396431b1-5fb2-4052-9290-13e14485008e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:53.773895 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396431b1-5fb2-4052-9290-13e14485008e service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:53.774178 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396431b1-5fb2-4052-9290-13e14485008e service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:53.774483 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396431b1-5fb2-4052-9290-13e14485008e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:53.776147 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-396431b1-5fb2-4052-9290-13e14485008e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:53.839243 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:53.902803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396431b1-5fb2-4052-9290-13e14485008e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:53.904260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396431b1-5fb2-4052-9290-13e14485008e service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:53.925670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 663/1311] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:22:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:22:56.277388 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5088a-3d9c-4952-8006-f1550de8b309 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:56.277996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5088a-3d9c-4952-8006-f1550de8b309 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:56.277996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5088a-3d9c-4952-8006-f1550de8b309 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:56.279632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 649/1312] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:22:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:56.286206 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01ec04-f79a-4f41-8333-be3087960ee3 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:56.286912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01ec04-f79a-4f41-8333-be3087960ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:56.286912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01ec04-f79a-4f41-8333-be3087960ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:56.365313 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-7d01ec04-f79a-4f41-8333-be3087960ee3 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:56.366544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-7d01ec04-f79a-4f41-8333-be3087960ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:56.474797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 664/1313] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:22:56.513719 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5088a-3d9c-4952-8006-f1550de8b309 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:56.515518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ffd5088a-3d9c-4952-8006-f1550de8b309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:56.641718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5088a-3d9c-4952-8006-f1550de8b309 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:56.722232 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:56.724924 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d37ec6-f4b3-40bc-a19f-59ddb44273b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:56.725592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d37ec6-f4b3-40bc-a19f-59ddb44273b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:56.725879 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d37ec6-f4b3-40bc-a19f-59ddb44273b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:56.726158 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d37ec6-f4b3-40bc-a19f-59ddb44273b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:56.728581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d8d37ec6-f4b3-40bc-a19f-59ddb44273b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:56.851209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d37ec6-f4b3-40bc-a19f-59ddb44273b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:56.852785 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d37ec6-f4b3-40bc-a19f-59ddb44273b7 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:22:56.873635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 650/1314] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:22:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:22:59.452590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887153d-db3d-4787-88c0-41f6980d758f None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:59.453092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887153d-db3d-4787-88c0-41f6980d758f None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:59.453092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887153d-db3d-4787-88c0-41f6980d758f None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:59.454450 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 665/1315] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:22:59.459516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f0c67-2715-4208-b55d-b0b6f04e3d6a None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:59.459926 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f0c67-2715-4208-b55d-b0b6f04e3d6a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:59.460078 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f0c67-2715-4208-b55d-b0b6f04e3d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:59.525627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-fb6f0c67-2715-4208-b55d-b0b6f04e3d6a None None] MFA Rules not processed for user `83a2209eb05748b8a18c53a472626c39`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:22:59.526870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-fb6f0c67-2715-4208-b55d-b0b6f04e3d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:59.578117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:59.623061 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 651/1316] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:22:59 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:22:59.663000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887153d-db3d-4787-88c0-41f6980d758f None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:22:59.664540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-7887153d-db3d-4787-88c0-41f6980d758f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:59.675257 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:59.776003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887153d-db3d-4787-88c0-41f6980d758f None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:22:59.867595 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cee982-7068-478a-a6fc-a6e99edd57d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:22:59.868310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cee982-7068-478a-a6fc-a6e99edd57d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:22:59.869275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cee982-7068-478a-a6fc-a6e99edd57d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:22:59.869550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cee982-7068-478a-a6fc-a6e99edd57d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:22:59.871303 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-81cee982-7068-478a-a6fc-a6e99edd57d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:22:59.934801 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:59.980628 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cee982-7068-478a-a6fc-a6e99edd57d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:22:59.982088 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cee982-7068-478a-a6fc-a6e99edd57d2 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:00.001661 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 666/1317] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:22:59 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:23:03.226373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7931a6e-2c29-4ead-8aa7-c7f9200864ce None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:03.226932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7931a6e-2c29-4ead-8aa7-c7f9200864ce None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:03.226932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7931a6e-2c29-4ead-8aa7-c7f9200864ce None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:03.228903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 652/1318] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:23:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:03.234443 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81955a-af4e-43fa-9898-bc935ef131d1 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:03.235139 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81955a-af4e-43fa-9898-bc935ef131d1 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:03.235265 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81955a-af4e-43fa-9898-bc935ef131d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:03.301452 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-1b81955a-af4e-43fa-9898-bc935ef131d1 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:03.302720 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1b81955a-af4e-43fa-9898-bc935ef131d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:03.400765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 667/1319] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:03.440158 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7931a6e-2c29-4ead-8aa7-c7f9200864ce None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:03.441706 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a7931a6e-2c29-4ead-8aa7-c7f9200864ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:03.545492 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7931a6e-2c29-4ead-8aa7-c7f9200864ce None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:03.621117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1178d-d370-4eb6-9cfe-8459b0943dab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:03.627957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1178d-d370-4eb6-9cfe-8459b0943dab service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:03.627957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1178d-d370-4eb6-9cfe-8459b0943dab service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:03.627957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1178d-d370-4eb6-9cfe-8459b0943dab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:03.627957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-dde1178d-d370-4eb6-9cfe-8459b0943dab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:03.632176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:03.733713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde1178d-d370-4eb6-9cfe-8459b0943dab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:03.735373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde1178d-d370-4eb6-9cfe-8459b0943dab service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:03.754212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 653/1320] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:23:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:23:06.925197 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810d004-6ed7-4e60-b96d-e0fb6c3e0549 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:06.925704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810d004-6ed7-4e60-b96d-e0fb6c3e0549 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:06.925704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810d004-6ed7-4e60-b96d-e0fb6c3e0549 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:06.927628 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 668/1321] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:06.934913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c61ca-fbb0-45cf-a6bd-ff4d7e66995f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:06.935436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c61ca-fbb0-45cf-a6bd-ff4d7e66995f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:06.935901 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c61ca-fbb0-45cf-a6bd-ff4d7e66995f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:07.028840 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-dc8c61ca-fbb0-45cf-a6bd-ff4d7e66995f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:07.030451 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-dc8c61ca-fbb0-45cf-a6bd-ff4d7e66995f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:07.108747 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:07.158531 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 654/1322] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:07.198124 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8810d004-6ed7-4e60-b96d-e0fb6c3e0549 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:07.199910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8810d004-6ed7-4e60-b96d-e0fb6c3e0549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:07.223196 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:07.321675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8810d004-6ed7-4e60-b96d-e0fb6c3e0549 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:07.396411 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3949439-3e58-407f-8f6a-e0e14ea9b467 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:07.397068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3949439-3e58-407f-8f6a-e0e14ea9b467 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:07.397351 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3949439-3e58-407f-8f6a-e0e14ea9b467 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:07.397624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3949439-3e58-407f-8f6a-e0e14ea9b467 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:07.399306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e3949439-3e58-407f-8f6a-e0e14ea9b467 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:07.470049 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:07.507895 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3949439-3e58-407f-8f6a-e0e14ea9b467 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:07.509264 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3949439-3e58-407f-8f6a-e0e14ea9b467 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:07.528599 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 669/1323] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:23:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:23:09.642623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99e22e-b2fe-4803-aa48-9b9415f7467a None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:09.643127 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99e22e-b2fe-4803-aa48-9b9415f7467a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:09.643127 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99e22e-b2fe-4803-aa48-9b9415f7467a None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:09.644506 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 655/1324] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:09.649820 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0797c6-50c6-46cc-a364-bc11d50657f6 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:09.650875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0797c6-50c6-46cc-a364-bc11d50657f6 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:09.651402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0797c6-50c6-46cc-a364-bc11d50657f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:09.716625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-dd0797c6-50c6-46cc-a364-bc11d50657f6 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:09.717976 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-dd0797c6-50c6-46cc-a364-bc11d50657f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:09.816855 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 670/1325] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:09.854183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99e22e-b2fe-4803-aa48-9b9415f7467a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:09.855750 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0d99e22e-b2fe-4803-aa48-9b9415f7467a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:09.963021 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99e22e-b2fe-4803-aa48-9b9415f7467a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:10.045020 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5b441-ccda-4d9f-adf3-af59d8676add service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:10.045679 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5b441-ccda-4d9f-adf3-af59d8676add service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:10.045967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5b441-ccda-4d9f-adf3-af59d8676add service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:10.046255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5b441-ccda-4d9f-adf3-af59d8676add service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:10.047947 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-2eb5b441-ccda-4d9f-adf3-af59d8676add service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:10.066574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:10.171174 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5b441-ccda-4d9f-adf3-af59d8676add service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:10.172615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5b441-ccda-4d9f-adf3-af59d8676add service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:10.199259 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 656/1326] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:23:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:23:12.352917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3468ce-aec4-4b34-9ad2-a5606140c2a5 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:12.353993 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3468ce-aec4-4b34-9ad2-a5606140c2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:12.353993 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3468ce-aec4-4b34-9ad2-a5606140c2a5 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:12.355437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 671/1327] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:23:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:12.361570 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfe5e1-ef92-4f2a-9bb8-ab3efc7b2eb7 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:12.362048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfe5e1-ef92-4f2a-9bb8-ab3efc7b2eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:12.362164 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfe5e1-ef92-4f2a-9bb8-ab3efc7b2eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:12.433665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-5ecfe5e1-ef92-4f2a-9bb8-ab3efc7b2eb7 None None] MFA Rules not processed for user `83a2209eb05748b8a18c53a472626c39`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:12.434946 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5ecfe5e1-ef92-4f2a-9bb8-ab3efc7b2eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:12.514986 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:12.549096 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 657/1328] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:12 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:12.604153 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37543ec7-e84e-4f97-a4ab-b901d4b9baf4 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:12.604356 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37543ec7-e84e-4f97-a4ab-b901d4b9baf4 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:12.604779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37543ec7-e84e-4f97-a4ab-b901d4b9baf4 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:12.606510 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 672/1329] 10.222.0.78 () {54 vars in 965 bytes} [Wed Jul 20 15:23:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:12.613615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbe83f-b4c6-43c3-8ec5-0740df962af7 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:12.613963 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbe83f-b4c6-43c3-8ec5-0740df962af7 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:12.614528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbe83f-b4c6-43c3-8ec5-0740df962af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:12.697960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-77dbe83f-b4c6-43c3-8ec5-0740df962af7 None None] MFA Rules not processed for user `b57530f18d9c4d328f8014db736d8de9`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:12.699210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-77dbe83f-b4c6-43c3-8ec5-0740df962af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:12.778719 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 658/1330] 10.222.0.78 () {58 vars in 1062 bytes} [Wed Jul 20 15:23:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:12.789008 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37543ec7-e84e-4f97-a4ab-b901d4b9baf4 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:12.790608 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-37543ec7-e84e-4f97-a4ab-b901d4b9baf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:12.821625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:12.909631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37543ec7-e84e-4f97-a4ab-b901d4b9baf4 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:12.993505 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df64da3e-352f-4b60-b3ee-e45205c63b94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:12.994705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64da3e-352f-4b60-b3ee-e45205c63b94 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:12.994989 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64da3e-352f-4b60-b3ee-e45205c63b94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:12.995263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64da3e-352f-4b60-b3ee-e45205c63b94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:12.996888 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-df64da3e-352f-4b60-b3ee-e45205c63b94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:13.075315 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:13.108104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df64da3e-352f-4b60-b3ee-e45205c63b94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:13.109722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df64da3e-352f-4b60-b3ee-e45205c63b94 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:13.128420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 673/1331] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:23:12 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:23:16.357589 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f3144-ef28-420a-b216-c266c2609354 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:16.358115 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f3144-ef28-420a-b216-c266c2609354 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:16.358115 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f3144-ef28-420a-b216-c266c2609354 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:16.359729 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 659/1332] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:16.364897 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5de59a-debd-4f17-91d7-67a8a1f779b5 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:16.365324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5de59a-debd-4f17-91d7-67a8a1f779b5 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:16.365476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5de59a-debd-4f17-91d7-67a8a1f779b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:16.431886 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-ba5de59a-debd-4f17-91d7-67a8a1f779b5 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:16.433276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ba5de59a-debd-4f17-91d7-67a8a1f779b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:16.533129 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 674/1333] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:16.575150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f3144-ef28-420a-b216-c266c2609354 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:16.576760 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0b7f3144-ef28-420a-b216-c266c2609354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:16.655292 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:16.696505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f3144-ef28-420a-b216-c266c2609354 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:16.779333 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef64ca8-33f2-4dec-9de5-7db56f8fc389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:16.779863 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef64ca8-33f2-4dec-9de5-7db56f8fc389 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:16.780145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef64ca8-33f2-4dec-9de5-7db56f8fc389 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:16.780418 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef64ca8-33f2-4dec-9de5-7db56f8fc389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:16.782727 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-3ef64ca8-33f2-4dec-9de5-7db56f8fc389 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:16.902566 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef64ca8-33f2-4dec-9de5-7db56f8fc389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:16.903945 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef64ca8-33f2-4dec-9de5-7db56f8fc389 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:16.919216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:16.924464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 660/1334] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:23:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:23:19.093905 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a02e8-316b-44c1-b7b8-7c027eafece8 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:19.094391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a02e8-316b-44c1-b7b8-7c027eafece8 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:19.094528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a02e8-316b-44c1-b7b8-7c027eafece8 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:19.095918 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 675/1335] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:19.101688 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de6519-4d6c-489d-8d63-aa7cdb65f156 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:19.102277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de6519-4d6c-489d-8d63-aa7cdb65f156 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:19.102277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de6519-4d6c-489d-8d63-aa7cdb65f156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:19.172031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-47de6519-4d6c-489d-8d63-aa7cdb65f156 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:19.173294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-47de6519-4d6c-489d-8d63-aa7cdb65f156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:19.281319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 661/1336] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:19.323087 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a02e8-316b-44c1-b7b8-7c027eafece8 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:19.324641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d34a02e8-316b-44c1-b7b8-7c027eafece8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:19.367563 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:19.434421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a02e8-316b-44c1-b7b8-7c027eafece8 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:19.512377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85351923-4f76-4af4-9e14-e5ce7b23df3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:19.513067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85351923-4f76-4af4-9e14-e5ce7b23df3e service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:19.513352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85351923-4f76-4af4-9e14-e5ce7b23df3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:19.513637 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85351923-4f76-4af4-9e14-e5ce7b23df3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:19.515375 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-85351923-4f76-4af4-9e14-e5ce7b23df3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:19.604956 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:19.632583 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85351923-4f76-4af4-9e14-e5ce7b23df3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:19.635643 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85351923-4f76-4af4-9e14-e5ce7b23df3e service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:19.663912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 676/1337] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:23:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:23:22.517691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed720d5-9381-4183-b3ec-8b94ddbbeafb None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:22.518224 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed720d5-9381-4183-b3ec-8b94ddbbeafb None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:22.518224 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed720d5-9381-4183-b3ec-8b94ddbbeafb None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:22.519650 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 662/1338] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:22.525131 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d703cb-1a0d-433d-a872-8415a9a93d5a None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:22.525675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d703cb-1a0d-433d-a872-8415a9a93d5a None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:22.525675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d703cb-1a0d-433d-a872-8415a9a93d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:22.599395 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-01d703cb-1a0d-433d-a872-8415a9a93d5a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:22.600606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-01d703cb-1a0d-433d-a872-8415a9a93d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:22.704438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 677/1339] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:22.743601 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed720d5-9381-4183-b3ec-8b94ddbbeafb None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:22.745149 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-aed720d5-9381-4183-b3ec-8b94ddbbeafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:22.827581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:22.863567 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed720d5-9381-4183-b3ec-8b94ddbbeafb None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:22.949124 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a709e94-2372-4dbf-aea4-d83ec76d3fbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:22.949788 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a709e94-2372-4dbf-aea4-d83ec76d3fbe service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:22.950071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a709e94-2372-4dbf-aea4-d83ec76d3fbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:22.950378 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a709e94-2372-4dbf-aea4-d83ec76d3fbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:22.952104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5a709e94-2372-4dbf-aea4-d83ec76d3fbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:23.067387 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a709e94-2372-4dbf-aea4-d83ec76d3fbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:23.068984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a709e94-2372-4dbf-aea4-d83ec76d3fbe service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:23.090339 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 663/1340] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:23:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:23:27.469604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b63fc-f19f-4908-9574-7dde0e819870 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:27.470079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b63fc-f19f-4908-9574-7dde0e819870 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:27.470190 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b63fc-f19f-4908-9574-7dde0e819870 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:27.471658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 678/1341] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:27.477127 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1478a0-d59b-4953-80cb-21b5a0bdebe3 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:27.477611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1478a0-d59b-4953-80cb-21b5a0bdebe3 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:27.478297 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1478a0-d59b-4953-80cb-21b5a0bdebe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:27.487299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:27.550231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-2d1478a0-d59b-4953-80cb-21b5a0bdebe3 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:27.551632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-2d1478a0-d59b-4953-80cb-21b5a0bdebe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:27.658390 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 664/1342] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:27.698282 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b63fc-f19f-4908-9574-7dde0e819870 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:27.699891 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-3d8b63fc-f19f-4908-9574-7dde0e819870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:27.749889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:27.804587 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b63fc-f19f-4908-9574-7dde0e819870 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:27.880101 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf70570-9b0e-4fbf-8953-e71118da1cce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:27.880749 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf70570-9b0e-4fbf-8953-e71118da1cce service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:27.881048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf70570-9b0e-4fbf-8953-e71118da1cce service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:27.881299 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf70570-9b0e-4fbf-8953-e71118da1cce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:27.882985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2bf70570-9b0e-4fbf-8953-e71118da1cce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:27.978002 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:27.994718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf70570-9b0e-4fbf-8953-e71118da1cce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:27.996166 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf70570-9b0e-4fbf-8953-e71118da1cce service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:28.015631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 679/1343] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:23:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:23:31.147355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3fae3-b960-4707-bcdf-fc3ecbf733db None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:31.147938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3fae3-b960-4707-bcdf-fc3ecbf733db None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:31.147938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3fae3-b960-4707-bcdf-fc3ecbf733db None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:31.149352 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 665/1344] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:31.154503 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf82d27-ae77-4509-87fd-c565af4da871 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:31.154942 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf82d27-ae77-4509-87fd-c565af4da871 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:31.155053 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf82d27-ae77-4509-87fd-c565af4da871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:31.223040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-fcf82d27-ae77-4509-87fd-c565af4da871 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:31.224264 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-fcf82d27-ae77-4509-87fd-c565af4da871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:31.325140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 680/1345] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:31 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:31.362696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3fae3-b960-4707-bcdf-fc3ecbf733db None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:31.364218 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-10e3fae3-b960-4707-bcdf-fc3ecbf733db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:31.461574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:31.480323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3fae3-b960-4707-bcdf-fc3ecbf733db None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:31.559908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15676fac-f403-4262-806a-4e1a2d965835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:31.560578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15676fac-f403-4262-806a-4e1a2d965835 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:31.560846 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15676fac-f403-4262-806a-4e1a2d965835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:31.561121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15676fac-f403-4262-806a-4e1a2d965835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:31.562780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-15676fac-f403-4262-806a-4e1a2d965835 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:31.672252 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15676fac-f403-4262-806a-4e1a2d965835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:31.673680 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15676fac-f403-4262-806a-4e1a2d965835 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:31.693282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 666/1346] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:23:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:23:36.084494 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87ce2a-d4c4-4538-963d-2b7b2651815e None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:36.085058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87ce2a-d4c4-4538-963d-2b7b2651815e None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:36.085058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87ce2a-d4c4-4538-963d-2b7b2651815e None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:36.086636 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 681/1347] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:36.094072 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bc787-3696-4de3-9f74-d62c1a8d04d8 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:36.094489 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bc787-3696-4de3-9f74-d62c1a8d04d8 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:36.094596 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bc787-3696-4de3-9f74-d62c1a8d04d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:36.110217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:36.170911 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-ca0bc787-3696-4de3-9f74-d62c1a8d04d8 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:36.172583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ca0bc787-3696-4de3-9f74-d62c1a8d04d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:36.277091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 667/1348] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:36.316820 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87ce2a-d4c4-4538-963d-2b7b2651815e None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:36.318438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2e87ce2a-d4c4-4538-963d-2b7b2651815e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:36.380357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:36.434769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87ce2a-d4c4-4538-963d-2b7b2651815e None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:36.514026 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da96cbe-d518-4f02-8764-50b3070944f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:36.514704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da96cbe-d518-4f02-8764-50b3070944f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:36.514981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da96cbe-d518-4f02-8764-50b3070944f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:36.515251 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da96cbe-d518-4f02-8764-50b3070944f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:36.516892 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1da96cbe-d518-4f02-8764-50b3070944f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:36.629379 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:36.633304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da96cbe-d518-4f02-8764-50b3070944f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:36.635426 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da96cbe-d518-4f02-8764-50b3070944f7 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:36.655202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 682/1349] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:23:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:23:41.769202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35627700-cf01-433e-89cc-73ce6d98ebcb None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:41.769756 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35627700-cf01-433e-89cc-73ce6d98ebcb None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:41.770539 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35627700-cf01-433e-89cc-73ce6d98ebcb None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:41.772408 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 668/1350] 10.222.0.78 () {54 vars in 921 bytes} [Wed Jul 20 15:23:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:41.778583 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06aebe2-94eb-47b3-8a0e-5a57b3404364 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:41.779420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06aebe2-94eb-47b3-8a0e-5a57b3404364 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:41.780037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06aebe2-94eb-47b3-8a0e-5a57b3404364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:41.952114 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-a06aebe2-94eb-47b3-8a0e-5a57b3404364 None None] MFA Rules not processed for user `2192547babde4449a46f9cb37ed64524`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:41.953312 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-a06aebe2-94eb-47b3-8a0e-5a57b3404364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:42.095830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 683/1351] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Jul 20 15:23:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:46.275288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=59 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:46.488759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad11eb-1b54-4b25-b49c-03cfe465094f None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:46.489355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad11eb-1b54-4b25-b49c-03cfe465094f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:46.489777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad11eb-1b54-4b25-b49c-03cfe465094f None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:46.491696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 669/1352] 10.222.0.78 () {54 vars in 921 bytes} [Wed Jul 20 15:23:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:46.498030 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc336cea-0c97-4831-8e99-a411797c8e06 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:46.498665 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc336cea-0c97-4831-8e99-a411797c8e06 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:46.498899 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc336cea-0c97-4831-8e99-a411797c8e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:46.560137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:46.583614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-bc336cea-0c97-4831-8e99-a411797c8e06 None None] MFA Rules not processed for user `2192547babde4449a46f9cb37ed64524`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:46.585059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-bc336cea-0c97-4831-8e99-a411797c8e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:46.701810 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 684/1353] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Jul 20 15:23:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:47.727429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417133d0-175a-46f1-99ab-c540bdb0c2de None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:47.727929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417133d0-175a-46f1-99ab-c540bdb0c2de None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:47.727929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417133d0-175a-46f1-99ab-c540bdb0c2de None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:47.729421 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 670/1354] 10.222.0.78 () {54 vars in 921 bytes} [Wed Jul 20 15:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:47.734690 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0236a-0a75-43a6-afa8-505b143967ed None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:47.735151 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0236a-0a75-43a6-afa8-505b143967ed None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:47.735268 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0236a-0a75-43a6-afa8-505b143967ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:47.813906 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-84b0236a-0a75-43a6-afa8-505b143967ed None None] MFA Rules not processed for user `2192547babde4449a46f9cb37ed64524`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:47.815262 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-84b0236a-0a75-43a6-afa8-505b143967ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:47.858093 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:47.923431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 685/1355] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Jul 20 15:23:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:51.528889 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dca2d0-8673-42b3-94ab-001d69eda82d None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:51.531091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dca2d0-8673-42b3-94ab-001d69eda82d None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:51.531552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dca2d0-8673-42b3-94ab-001d69eda82d None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:51.533582 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 671/1356] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:23:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:51.540668 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc05e1b-cb87-4893-8cc3-980538eafb36 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:51.541679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc05e1b-cb87-4893-8cc3-980538eafb36 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:51.542290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc05e1b-cb87-4893-8cc3-980538eafb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:51.637791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-1fc05e1b-cb87-4893-8cc3-980538eafb36 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:51.639214 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1fc05e1b-cb87-4893-8cc3-980538eafb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:51.745409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 686/1357] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:51.802505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476ca09-8b43-4f42-ab89-e84995d7f637 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:51.803069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476ca09-8b43-4f42-ab89-e84995d7f637 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:51.803725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476ca09-8b43-4f42-ab89-e84995d7f637 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:51.805376 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 672/1358] 10.222.0.78 () {54 vars in 962 bytes} [Wed Jul 20 15:23:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:51.814079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d7141-23ee-4db1-b688-3522e42c7b74 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:51.814665 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d7141-23ee-4db1-b688-3522e42c7b74 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:51.815152 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d7141-23ee-4db1-b688-3522e42c7b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:51.832991 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:51.903117 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-ff0d7141-23ee-4db1-b688-3522e42c7b74 None None] MFA Rules not processed for user `a4a24d5a375945779fa626fae25ac237`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:51.904339 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ff0d7141-23ee-4db1-b688-3522e42c7b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:52.028673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 687/1359] 10.222.0.78 () {58 vars in 1059 bytes} [Wed Jul 20 15:23:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:23:52.036825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476ca09-8b43-4f42-ab89-e84995d7f637 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:52.038439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4476ca09-8b43-4f42-ab89-e84995d7f637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:52.203592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476ca09-8b43-4f42-ab89-e84995d7f637 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:52.278878 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:52.352849 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551f76a-4533-41a1-af43-110ce302a503 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:52.353500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551f76a-4533-41a1-af43-110ce302a503 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:52.353819 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551f76a-4533-41a1-af43-110ce302a503 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:52.354329 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551f76a-4533-41a1-af43-110ce302a503 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:52.356065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-9551f76a-4533-41a1-af43-110ce302a503 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:52.458286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551f76a-4533-41a1-af43-110ce302a503 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:52.460039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551f76a-4533-41a1-af43-110ce302a503 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:52.480488 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 673/1360] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:23:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:23:54.767391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b1a69-89e5-4764-ba36-e58a7bcfc4b2 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:54.767816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b1a69-89e5-4764-ba36-e58a7bcfc4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:54.768012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b1a69-89e5-4764-ba36-e58a7bcfc4b2 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:54.769696 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 688/1361] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:54.786110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a7462-0977-4c5a-9ef4-3d0c0244ef1c None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:54.786110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a7462-0977-4c5a-9ef4-3d0c0244ef1c None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:54.786110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a7462-0977-4c5a-9ef4-3d0c0244ef1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:54.866727 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-ca9a7462-0977-4c5a-9ef4-3d0c0244ef1c None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:54.868824 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ca9a7462-0977-4c5a-9ef4-3d0c0244ef1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:54.915315 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:54.990412 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 674/1362] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:55.035203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b1a69-89e5-4764-ba36-e58a7bcfc4b2 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:55.036654 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-470b1a69-89e5-4764-ba36-e58a7bcfc4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:55.120018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:55.225664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b1a69-89e5-4764-ba36-e58a7bcfc4b2 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:55.390335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c770b32-087d-488d-bd2d-e1459ee7335c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:55.390976 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c770b32-087d-488d-bd2d-e1459ee7335c service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:55.391280 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c770b32-087d-488d-bd2d-e1459ee7335c service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:55.391596 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c770b32-087d-488d-bd2d-e1459ee7335c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:55.393509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-4c770b32-087d-488d-bd2d-e1459ee7335c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:55.435091 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:55.520457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c770b32-087d-488d-bd2d-e1459ee7335c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:55.522516 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c770b32-087d-488d-bd2d-e1459ee7335c service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:55.550962 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 689/1363] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:23:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:23:58.116108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e948d-55cd-4330-bf33-b50b87c0f5b7 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:58.116618 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e948d-55cd-4330-bf33-b50b87c0f5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:58.116755 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e948d-55cd-4330-bf33-b50b87c0f5b7 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:58.118205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 675/1364] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:58.123842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582ca96-7f65-4ce7-995b-b35e73478fe4 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:58.124309 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582ca96-7f65-4ce7-995b-b35e73478fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:58.124404 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582ca96-7f65-4ce7-995b-b35e73478fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:58.199174 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-d582ca96-7f65-4ce7-995b-b35e73478fe4 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:58.200588 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d582ca96-7f65-4ce7-995b-b35e73478fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:58.315230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:58.318055 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 690/1365] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:58.366474 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b3aa8-629b-44e7-acdb-ee9695970eeb None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:58.366764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b3aa8-629b-44e7-acdb-ee9695970eeb None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:58.367048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b3aa8-629b-44e7-acdb-ee9695970eeb None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:58.368426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 676/1366] 10.222.0.78 () {54 vars in 962 bytes} [Wed Jul 20 15:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:23:58.377652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b207de3-1232-498d-85ef-61a7026c6c87 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:58.377946 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b207de3-1232-498d-85ef-61a7026c6c87 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:58.378227 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b207de3-1232-498d-85ef-61a7026c6c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:58.459391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-7b207de3-1232-498d-85ef-61a7026c6c87 None None] MFA Rules not processed for user `a4a24d5a375945779fa626fae25ac237`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:23:58.461332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-7b207de3-1232-498d-85ef-61a7026c6c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:58.586711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 691/1367] 10.222.0.78 () {58 vars in 1059 bytes} [Wed Jul 20 15:23:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:23:58.594582 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b3aa8-629b-44e7-acdb-ee9695970eeb None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:23:58.596128 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-da5b3aa8-629b-44e7-acdb-ee9695970eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:58.870563 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b3aa8-629b-44e7-acdb-ee9695970eeb None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:23:58.892899 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:59.009785 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495146ff-28b3-4495-a0e6-d446a23df2cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:23:59.010473 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495146ff-28b3-4495-a0e6-d446a23df2cf service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:59.010763 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495146ff-28b3-4495-a0e6-d446a23df2cf service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:59.011043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495146ff-28b3-4495-a0e6-d446a23df2cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:23:59.012717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-495146ff-28b3-4495-a0e6-d446a23df2cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:23:59.114151 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495146ff-28b3-4495-a0e6-d446a23df2cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:23:59.115788 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495146ff-28b3-4495-a0e6-d446a23df2cf service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:23:59.136717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 677/1368] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:23:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:23:59.998877 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92a423-3b8e-41d5-bcd6-d169661b0be6 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:23:59.999446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92a423-3b8e-41d5-bcd6-d169661b0be6 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:23:59.999446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92a423-3b8e-41d5-bcd6-d169661b0be6 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:00.000992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 692/1369] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:00.006141 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c1888-bd7f-4f03-b396-9f456ae4d170 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:00.006473 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c1888-bd7f-4f03-b396-9f456ae4d170 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:00.006750 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c1888-bd7f-4f03-b396-9f456ae4d170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:00.047936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:00.086953 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-bf7c1888-bd7f-4f03-b396-9f456ae4d170 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:00.088641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-bf7c1888-bd7f-4f03-b396-9f456ae4d170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:00.215744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 678/1370] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:00.529601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8b44a-b106-4111-8a2c-afe3822d5657 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:00.530012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8b44a-b106-4111-8a2c-afe3822d5657 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:00.530465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8b44a-b106-4111-8a2c-afe3822d5657 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:00.531930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 693/1371] 10.222.0.78 () {54 vars in 964 bytes} [Wed Jul 20 15:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:00.537086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5813b0e-78f9-428e-be2f-3a4fede1f8b3 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:00.537385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5813b0e-78f9-428e-be2f-3a4fede1f8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:00.537671 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5813b0e-78f9-428e-be2f-3a4fede1f8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:00.617699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-b5813b0e-78f9-428e-be2f-3a4fede1f8b3 None None] MFA Rules not processed for user `b51e9bdab4164d87b7b0b11c505c2c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:00.618947 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b5813b0e-78f9-428e-be2f-3a4fede1f8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:00.667254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:00.705648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 679/1372] 10.222.0.78 () {58 vars in 1061 bytes} [Wed Jul 20 15:24:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:00.712842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8b44a-b106-4111-8a2c-afe3822d5657 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:00.714933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-bfa8b44a-b106-4111-8a2c-afe3822d5657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:00.771455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:00.835237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8b44a-b106-4111-8a2c-afe3822d5657 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:00.924567 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1e2f6-0813-4020-b0c6-18c442419a1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:00.925222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1e2f6-0813-4020-b0c6-18c442419a1f service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:00.925503 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1e2f6-0813-4020-b0c6-18c442419a1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:00.925777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1e2f6-0813-4020-b0c6-18c442419a1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:00.927462 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-a7c1e2f6-0813-4020-b0c6-18c442419a1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:01.044626 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:01.059141 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1e2f6-0813-4020-b0c6-18c442419a1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:01.060773 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1e2f6-0813-4020-b0c6-18c442419a1f service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:01.082429 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 694/1373] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:01.437013 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e56b65-b11c-426d-aea9-70fb018ef204 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:01.437512 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e56b65-b11c-426d-aea9-70fb018ef204 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:01.437757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e56b65-b11c-426d-aea9-70fb018ef204 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:01.439171 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 680/1374] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:01.444433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ed32b-26dd-42fe-94f0-3eda12e9e1a5 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:01.444808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ed32b-26dd-42fe-94f0-3eda12e9e1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:01.445688 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ed32b-26dd-42fe-94f0-3eda12e9e1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:01.519706 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-001ed32b-26dd-42fe-94f0-3eda12e9e1a5 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:01.520932 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-001ed32b-26dd-42fe-94f0-3eda12e9e1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:01.637941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 695/1375] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:24:01.682830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e56b65-b11c-426d-aea9-70fb018ef204 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:01.684420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-23e56b65-b11c-426d-aea9-70fb018ef204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:01.919148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e56b65-b11c-426d-aea9-70fb018ef204 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:01.919148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:01.978883 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532815cf-6a4f-4849-adc6-e09d0cf11119 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:01.996739 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532815cf-6a4f-4849-adc6-e09d0cf11119 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:01.996739 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532815cf-6a4f-4849-adc6-e09d0cf11119 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:01.996739 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532815cf-6a4f-4849-adc6-e09d0cf11119 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:01.996739 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-532815cf-6a4f-4849-adc6-e09d0cf11119 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:02.086691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532815cf-6a4f-4849-adc6-e09d0cf11119 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:02.088367 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532815cf-6a4f-4849-adc6-e09d0cf11119 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:02.108897 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 681/1376] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:04.339183 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab82e26-1f55-4ea4-b4c4-18c512bc9625 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:04.339745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab82e26-1f55-4ea4-b4c4-18c512bc9625 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:04.339745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab82e26-1f55-4ea4-b4c4-18c512bc9625 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:04.341139 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 696/1377] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:04.346785 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c89e09-8bfb-4996-8aed-0844d734bb8c None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:04.347244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c89e09-8bfb-4996-8aed-0844d734bb8c None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:04.347355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c89e09-8bfb-4996-8aed-0844d734bb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:04.420640 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-c9c89e09-8bfb-4996-8aed-0844d734bb8c None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:04.421877 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-c9c89e09-8bfb-4996-8aed-0844d734bb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:04.430277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:04.544794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 682/1378] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:04.599664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab82e26-1f55-4ea4-b4c4-18c512bc9625 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:04.600989 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-bab82e26-1f55-4ea4-b4c4-18c512bc9625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:04.678360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:04.791001 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab82e26-1f55-4ea4-b4c4-18c512bc9625 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:04.929787 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242572a-3937-4ac9-a63f-eddef6c14568 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:04.929787 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242572a-3937-4ac9-a63f-eddef6c14568 service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:04.930206 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242572a-3937-4ac9-a63f-eddef6c14568 service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:04.930661 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242572a-3937-4ac9-a63f-eddef6c14568 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:04.932469 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-4242572a-3937-4ac9-a63f-eddef6c14568 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:04.960995 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:05.053896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242572a-3937-4ac9-a63f-eddef6c14568 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:05.056395 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242572a-3937-4ac9-a63f-eddef6c14568 service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:05.081431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 697/1379] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:07.387742 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db857c74-3bb9-4bc0-a8d7-08e64fb69ffa None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:07.388321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db857c74-3bb9-4bc0-a8d7-08e64fb69ffa None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:07.388321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db857c74-3bb9-4bc0-a8d7-08e64fb69ffa None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:07.390057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 683/1380] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:07.395216 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95297ad6-3078-4406-81f2-547eaea97bc8 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:07.395710 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95297ad6-3078-4406-81f2-547eaea97bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:07.395817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95297ad6-3078-4406-81f2-547eaea97bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:07.468250 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-95297ad6-3078-4406-81f2-547eaea97bc8 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:07.469505 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-95297ad6-3078-4406-81f2-547eaea97bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:07.593087 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:07.606304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 698/1381] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:07.646028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db857c74-3bb9-4bc0-a8d7-08e64fb69ffa None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:07.647564 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-db857c74-3bb9-4bc0-a8d7-08e64fb69ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:07.806957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:07.817907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db857c74-3bb9-4bc0-a8d7-08e64fb69ffa None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:07.942991 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac117d9-4b1a-4385-889f-84f3ea66c3cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:07.943565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac117d9-4b1a-4385-889f-84f3ea66c3cb service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:07.943868 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac117d9-4b1a-4385-889f-84f3ea66c3cb service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:07.944182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac117d9-4b1a-4385-889f-84f3ea66c3cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:07.945977 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-3ac117d9-4b1a-4385-889f-84f3ea66c3cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:08.055581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac117d9-4b1a-4385-889f-84f3ea66c3cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:08.057495 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac117d9-4b1a-4385-889f-84f3ea66c3cb service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:08.071553 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:08.081955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 684/1382] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:10.331422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fc932-c71e-49d8-9984-c33438b8e824 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:10.331950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fc932-c71e-49d8-9984-c33438b8e824 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:10.332067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fc932-c71e-49d8-9984-c33438b8e824 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:10.333467 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 699/1383] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:10.338915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffb1e4-22ea-424d-847d-14c4c60e72bd None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:10.339363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffb1e4-22ea-424d-847d-14c4c60e72bd None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:10.339460 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffb1e4-22ea-424d-847d-14c4c60e72bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:10.416995 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-e7ffb1e4-22ea-424d-847d-14c4c60e72bd None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:10.418601 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e7ffb1e4-22ea-424d-847d-14c4c60e72bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:10.522185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 685/1384] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:10.566579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fc932-c71e-49d8-9984-c33438b8e824 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:10.568433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2b0fc932-c71e-49d8-9984-c33438b8e824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:10.746970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fc932-c71e-49d8-9984-c33438b8e824 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:10.819855 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:10.871073 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b164a-6f15-4718-a9a9-284d118f1329 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:10.871662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b164a-6f15-4718-a9a9-284d118f1329 service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:10.871981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b164a-6f15-4718-a9a9-284d118f1329 service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:10.872179 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b164a-6f15-4718-a9a9-284d118f1329 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:10.873839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-7f9b164a-6f15-4718-a9a9-284d118f1329 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:10.977723 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b164a-6f15-4718-a9a9-284d118f1329 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:10.979483 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b164a-6f15-4718-a9a9-284d118f1329 service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:10.998231 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 700/1385] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:13.301557 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6c45f-65e7-4956-8067-859a05db16a4 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:13.302243 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6c45f-65e7-4956-8067-859a05db16a4 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:13.302243 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6c45f-65e7-4956-8067-859a05db16a4 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:13.303515 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 686/1386] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:13.309285 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ee3c1-0300-4370-89e7-339b09388333 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:13.309724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ee3c1-0300-4370-89e7-339b09388333 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:13.309830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ee3c1-0300-4370-89e7-339b09388333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:13.382692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-2d6ee3c1-0300-4370-89e7-339b09388333 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:13.384320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2d6ee3c1-0300-4370-89e7-339b09388333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:13.427254 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:13.498128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 701/1387] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:13.543786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6c45f-65e7-4956-8067-859a05db16a4 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:13.545363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-eef6c45f-65e7-4956-8067-859a05db16a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:13.627408 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:13.706066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6c45f-65e7-4956-8067-859a05db16a4 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:13.829214 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6d707-1198-4b72-9776-e028502e46f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:13.829903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6d707-1198-4b72-9776-e028502e46f1 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:13.830291 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6d707-1198-4b72-9776-e028502e46f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:13.830609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6d707-1198-4b72-9776-e028502e46f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:13.832537 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-72d6d707-1198-4b72-9776-e028502e46f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:13.876314 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:13.932737 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6d707-1198-4b72-9776-e028502e46f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:13.934069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6d707-1198-4b72-9776-e028502e46f1 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:13.952457 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 687/1388] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:16.147037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c7567-3e55-4b3d-a861-67d0501348a4 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:16.148226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c7567-3e55-4b3d-a861-67d0501348a4 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:16.148226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c7567-3e55-4b3d-a861-67d0501348a4 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:16.149557 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 702/1389] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:24:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:16.155938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ee965-70ce-4c48-85df-22bee1d219f9 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:16.156691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ee965-70ce-4c48-85df-22bee1d219f9 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:16.156691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ee965-70ce-4c48-85df-22bee1d219f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:16.227803 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-1c4ee965-70ce-4c48-85df-22bee1d219f9 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:16.229013 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-1c4ee965-70ce-4c48-85df-22bee1d219f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:16.327473 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 688/1390] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:16.371766 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c7567-3e55-4b3d-a861-67d0501348a4 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:16.373322 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c48c7567-3e55-4b3d-a861-67d0501348a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:16.520867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c7567-3e55-4b3d-a861-67d0501348a4 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:16.537891 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:16.643791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2402116-6c65-48f7-ad89-6d6de1e8e198 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:16.644449 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2402116-6c65-48f7-ad89-6d6de1e8e198 service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:16.644732 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2402116-6c65-48f7-ad89-6d6de1e8e198 service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:16.645008 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2402116-6c65-48f7-ad89-6d6de1e8e198 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:16.646689 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b2402116-6c65-48f7-ad89-6d6de1e8e198 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:16.737925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2402116-6c65-48f7-ad89-6d6de1e8e198 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:16.739619 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2402116-6c65-48f7-ad89-6d6de1e8e198 service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:16.758921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 703/1391] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:17.092592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f491e6e-ecd5-4819-a390-c889748c75f6 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:17.092888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f491e6e-ecd5-4819-a390-c889748c75f6 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:17.093171 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f491e6e-ecd5-4819-a390-c889748c75f6 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:17.094874 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 689/1392] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:17.100232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c350924-a811-4c3f-9004-7345f7a53a55 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:17.101016 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c350924-a811-4c3f-9004-7345f7a53a55 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:17.101309 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c350924-a811-4c3f-9004-7345f7a53a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:17.127023 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:17.169114 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-4c350924-a811-4c3f-9004-7345f7a53a55 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:17.170945 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-4c350924-a811-4c3f-9004-7345f7a53a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:17.268752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 704/1393] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:17.327986 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55e5c5-457c-45cd-8f6d-864748f3023a None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:17.328433 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55e5c5-457c-45cd-8f6d-864748f3023a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:17.328549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55e5c5-457c-45cd-8f6d-864748f3023a None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:17.329891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 690/1394] 10.222.0.78 () {54 vars in 966 bytes} [Wed Jul 20 15:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:17.334940 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6bc78-8e54-499c-b179-e20dd8ac0872 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:17.335231 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6bc78-8e54-499c-b179-e20dd8ac0872 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:17.335516 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6bc78-8e54-499c-b179-e20dd8ac0872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:17.411226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-e7d6bc78-8e54-499c-b179-e20dd8ac0872 None None] MFA Rules not processed for user `b6c13596c2df438294bdac7c52ae8cb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:17.412442 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e7d6bc78-8e54-499c-b179-e20dd8ac0872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:17.454362 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:17.495196 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 705/1395] 10.222.0.78 () {58 vars in 1063 bytes} [Wed Jul 20 15:24:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:17.502833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55e5c5-457c-45cd-8f6d-864748f3023a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:17.504354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5e55e5c5-457c-45cd-8f6d-864748f3023a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:17.518529 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:17.612296 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55e5c5-457c-45cd-8f6d-864748f3023a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:17.686538 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc760e39-14ea-46c7-a1a9-8680fa541a29 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6c13596c2df438294bdac7c52ae8cb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:17.687343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc760e39-14ea-46c7-a1a9-8680fa541a29 service glance] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:17.687746 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc760e39-14ea-46c7-a1a9-8680fa541a29 service glance] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:17.688121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc760e39-14ea-46c7-a1a9-8680fa541a29 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:17.690043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-bc760e39-14ea-46c7-a1a9-8680fa541a29 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:17.760649 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:17.807793 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc760e39-14ea-46c7-a1a9-8680fa541a29 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:17.809504 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc760e39-14ea-46c7-a1a9-8680fa541a29 service glance] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:17.829163 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 691/1396] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:18.725422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fbc60-6c3f-4a1f-a7a7-8fa46e66034a None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:18.727288 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fbc60-6c3f-4a1f-a7a7-8fa46e66034a None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:18.727288 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fbc60-6c3f-4a1f-a7a7-8fa46e66034a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:18.799477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-b41fbc60-6c3f-4a1f-a7a7-8fa46e66034a None None] MFA Rules not processed for user `4e75a82cc7e94702ba0080fc65d6eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:18.800710 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b41fbc60-6c3f-4a1f-a7a7-8fa46e66034a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:18.901433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 706/1397] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Jul 20 15:24:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:18.941347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082b3b9-ebe1-41d2-b2a2-fb330c7e4bb7 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:18.941800 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082b3b9-ebe1-41d2-b2a2-fb330c7e4bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:18.942103 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082b3b9-ebe1-41d2-b2a2-fb330c7e4bb7 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:18.943311 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 692/1398] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:18.949709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5373bcf-37dc-4cfb-81ae-1b0ad928aa55 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:18.954913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5373bcf-37dc-4cfb-81ae-1b0ad928aa55 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:18.954913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5373bcf-37dc-4cfb-81ae-1b0ad928aa55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:18.955537 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6de588-17ae-4ebc-b698-23634583033f None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:18.955887 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6de588-17ae-4ebc-b698-23634583033f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:18.956229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6de588-17ae-4ebc-b698-23634583033f None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:18.957692 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 693/1399] 10.222.0.78 () {54 vars in 943 bytes} [Wed Jul 20 15:24:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:18.966697 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9a01f-7a1c-4aa7-b08c-5686ea37563b None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:18.966994 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9a01f-7a1c-4aa7-b08c-5686ea37563b None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:18.967272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9a01f-7a1c-4aa7-b08c-5686ea37563b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:19.022440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-a5373bcf-37dc-4cfb-81ae-1b0ad928aa55 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:19.023669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-a5373bcf-37dc-4cfb-81ae-1b0ad928aa55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:19.030364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:19.045108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-bac9a01f-7a1c-4aa7-b08c-5686ea37563b None None] MFA Rules not processed for user `ed0d719c0eb04a3b8a228c6ce70fdf98`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:19.047754 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-bac9a01f-7a1c-4aa7-b08c-5686ea37563b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:19.125241 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 707/1400] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:19.160352 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 694/1401] 10.222.0.78 () {58 vars in 1040 bytes} [Wed Jul 20 15:24:18 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:24:19.169851 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5373bcf-37dc-4cfb-81ae-1b0ad928aa55 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:19.171114 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-a5373bcf-37dc-4cfb-81ae-1b0ad928aa55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:19.171667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9a01f-7a1c-4aa7-b08c-5686ea37563b None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:19.172838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-bac9a01f-7a1c-4aa7-b08c-5686ea37563b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:19.176182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:19.275998 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9a01f-7a1c-4aa7-b08c-5686ea37563b None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:19.292809 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5373bcf-37dc-4cfb-81ae-1b0ad928aa55 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:19.342160 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:19.399532 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eaffa-9a08-4900-8d23-aa135309a532 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:19.400182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eaffa-9a08-4900-8d23-aa135309a532 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:19.400464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eaffa-9a08-4900-8d23-aa135309a532 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:19.400736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eaffa-9a08-4900-8d23-aa135309a532 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:19.402400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-329eaffa-9a08-4900-8d23-aa135309a532 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:19.416285 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:19.419946 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a5422-d8e2-439f-9283-71b181136940 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0d719c0eb04a3b8a228c6ce70fdf98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:19.420610 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a5422-d8e2-439f-9283-71b181136940 service swift] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:19.420891 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a5422-d8e2-439f-9283-71b181136940 service swift] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:19.421173 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a5422-d8e2-439f-9283-71b181136940 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:19.422882 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6a3a5422-d8e2-439f-9283-71b181136940 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:19.500683 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329eaffa-9a08-4900-8d23-aa135309a532 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:19.502378 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329eaffa-9a08-4900-8d23-aa135309a532 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:19.511792 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a5422-d8e2-439f-9283-71b181136940 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:19.513425 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a5422-d8e2-439f-9283-71b181136940 service swift] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:19.523692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 708/1402] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Jul 20 15:24:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:24:19.527234 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 695/1403] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:24:21.763436 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bd125-6715-4fa9-8975-6d3086d773ce None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:21.767098 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bd125-6715-4fa9-8975-6d3086d773ce None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:21.767098 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bd125-6715-4fa9-8975-6d3086d773ce None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:21.767098 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 709/1404] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:24:21.771231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f95b5-67e0-4b05-ac8e-e22eae21ee6e None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:21.771769 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f95b5-67e0-4b05-ac8e-e22eae21ee6e None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:21.771769 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f95b5-67e0-4b05-ac8e-e22eae21ee6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:21.848693 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-9b8f95b5-67e0-4b05-ac8e-e22eae21ee6e None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:21.849910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-9b8f95b5-67e0-4b05-ac8e-e22eae21ee6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:21.963257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:21.977477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 696/1405] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:22.074661 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bd125-6715-4fa9-8975-6d3086d773ce None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:22.074661 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c08bd125-6715-4fa9-8975-6d3086d773ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:22.153214 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:22.241211 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bd125-6715-4fa9-8975-6d3086d773ce None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:22.402229 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c22b8-07f5-4bc8-8ca5-7fcfc978787a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:22.402909 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c22b8-07f5-4bc8-8ca5-7fcfc978787a service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:22.403194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c22b8-07f5-4bc8-8ca5-7fcfc978787a service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:22.403465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c22b8-07f5-4bc8-8ca5-7fcfc978787a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:22.405131 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1b6c22b8-07f5-4bc8-8ca5-7fcfc978787a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:22.513289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c22b8-07f5-4bc8-8ca5-7fcfc978787a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:22.514946 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c22b8-07f5-4bc8-8ca5-7fcfc978787a service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:22.534289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 710/1406] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:22.689074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:24.917624 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099577d-343a-45b6-860c-51b7503f173c None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:24.918155 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099577d-343a-45b6-860c-51b7503f173c None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:24.918155 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099577d-343a-45b6-860c-51b7503f173c None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:24.923921 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 697/1407] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:24:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:24.925143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f9baf-6152-44b6-869e-6b8c74ffa1eb None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:24.925587 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f9baf-6152-44b6-869e-6b8c74ffa1eb None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:24.925698 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f9baf-6152-44b6-869e-6b8c74ffa1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:24.996804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-4b4f9baf-6152-44b6-869e-6b8c74ffa1eb None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:24.998053 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-4b4f9baf-6152-44b6-869e-6b8c74ffa1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:25.244595 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 711/1408] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:24:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:24:25.289939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099577d-343a-45b6-860c-51b7503f173c None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:25.291820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d099577d-343a-45b6-860c-51b7503f173c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:25.457239 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099577d-343a-45b6-860c-51b7503f173c None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:25.524723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:25.582055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6c868-4a86-42e5-a35f-2907140d3d8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:25.582756 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6c868-4a86-42e5-a35f-2907140d3d8d service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:25.583043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6c868-4a86-42e5-a35f-2907140d3d8d service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:25.583321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6c868-4a86-42e5-a35f-2907140d3d8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:25.585031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0ad6c868-4a86-42e5-a35f-2907140d3d8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:25.691710 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad6c868-4a86-42e5-a35f-2907140d3d8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:25.693414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad6c868-4a86-42e5-a35f-2907140d3d8d service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:25.712760 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 698/1409] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:35.242534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:35.709995 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=66 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:43.546583 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee419a2e-22a4-4247-b258-4f58007d2930 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:43.547156 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee419a2e-22a4-4247-b258-4f58007d2930 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:43.547156 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee419a2e-22a4-4247-b258-4f58007d2930 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:43.548517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 712/1410] 10.222.0.78 () {54 vars in 918 bytes} [Wed Jul 20 15:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:24:43.553842 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f61a52-d46c-4e47-92d8-791e3527876f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:43.554304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f61a52-d46c-4e47-92d8-791e3527876f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:43.554421 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f61a52-d46c-4e47-92d8-791e3527876f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:43.628930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-71f61a52-d46c-4e47-92d8-791e3527876f None None] MFA Rules not processed for user `2192547babde4449a46f9cb37ed64524`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:43.630051 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-71f61a52-d46c-4e47-92d8-791e3527876f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:43.737506 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 699/1411] 10.222.0.78 () {58 vars in 1015 bytes} [Wed Jul 20 15:24:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:43.753198 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055cd11-50ba-4686-8a36-91294c12cafc None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:43.753480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055cd11-50ba-4686-8a36-91294c12cafc None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:43.753754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055cd11-50ba-4686-8a36-91294c12cafc None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:43.755148 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 713/1412] 10.222.0.78 () {54 vars in 918 bytes} [Wed Jul 20 15:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:43.760963 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ebfc7-44eb-41eb-af14-c84e4d14109b None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:43.761250 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ebfc7-44eb-41eb-af14-c84e4d14109b None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:43.761528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ebfc7-44eb-41eb-af14-c84e4d14109b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:43.832826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:43.835281 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-d39ebfc7-44eb-41eb-af14-c84e4d14109b None None] MFA Rules not processed for user `a4a24d5a375945779fa626fae25ac237`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:43.836497 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d39ebfc7-44eb-41eb-af14-c84e4d14109b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:43.951687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 700/1413] 10.222.0.78 () {58 vars in 1015 bytes} [Wed Jul 20 15:24:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:46.495736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4445b6c-3a78-4217-bca0-806a0b68d140 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:46.496300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4445b6c-3a78-4217-bca0-806a0b68d140 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:46.496300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4445b6c-3a78-4217-bca0-806a0b68d140 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:46.498119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:24:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:46.503375 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2b665-87f9-43cf-98ed-600b0cc5a0ba None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:46.503812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2b665-87f9-43cf-98ed-600b0cc5a0ba None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:46.503963 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2b665-87f9-43cf-98ed-600b0cc5a0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:46.571975 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-f0e2b665-87f9-43cf-98ed-600b0cc5a0ba None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:46.573262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f0e2b665-87f9-43cf-98ed-600b0cc5a0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:46.631284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:46.677403 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:46.721249 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4445b6c-3a78-4217-bca0-806a0b68d140 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:46.722804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c4445b6c-3a78-4217-bca0-806a0b68d140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:46.875858 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4445b6c-3a78-4217-bca0-806a0b68d140 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:46.952798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:46.995765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77611071-77f1-4df8-bad1-8fff53fea787 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:46.996468 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77611071-77f1-4df8-bad1-8fff53fea787 service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:46.996800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77611071-77f1-4df8-bad1-8fff53fea787 service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:46.997135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77611071-77f1-4df8-bad1-8fff53fea787 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:46.998984 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-77611071-77f1-4df8-bad1-8fff53fea787 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:47.086188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77611071-77f1-4df8-bad1-8fff53fea787 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:47.087813 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77611071-77f1-4df8-bad1-8fff53fea787 service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:47.106129 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 715/1416] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:48.009951 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230d7d5-5afa-4ccf-91de-6b9d23c0ff90 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:48.010664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230d7d5-5afa-4ccf-91de-6b9d23c0ff90 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:48.010664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230d7d5-5afa-4ccf-91de-6b9d23c0ff90 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:48.011987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:48.017213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01e4ac-6511-463f-96b9-acf1f7559efc None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:48.017698 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01e4ac-6511-463f-96b9-acf1f7559efc None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:48.017796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01e4ac-6511-463f-96b9-acf1f7559efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:48.093052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-cb01e4ac-6511-463f-96b9-acf1f7559efc None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:48.094286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-cb01e4ac-6511-463f-96b9-acf1f7559efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:48.122024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:48.191698 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:48.342641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230d7d5-5afa-4ccf-91de-6b9d23c0ff90 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:48.344118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0230d7d5-5afa-4ccf-91de-6b9d23c0ff90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:48.493408 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230d7d5-5afa-4ccf-91de-6b9d23c0ff90 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:48.535721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:48.618972 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6d493-d07a-4b51-a4fa-e96ce94e4f84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:48.619527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6d493-d07a-4b51-a4fa-e96ce94e4f84 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:48.619806 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6d493-d07a-4b51-a4fa-e96ce94e4f84 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:48.620078 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6d493-d07a-4b51-a4fa-e96ce94e4f84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:48.621740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-67d6d493-d07a-4b51-a4fa-e96ce94e4f84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:48.730938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6d493-d07a-4b51-a4fa-e96ce94e4f84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:48.733822 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6d493-d07a-4b51-a4fa-e96ce94e4f84 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:48.757451 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 703/1419] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:49.669423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95913ec3-5891-4437-92dc-7e13a31d12af None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:49.669997 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95913ec3-5891-4437-92dc-7e13a31d12af None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:49.669997 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95913ec3-5891-4437-92dc-7e13a31d12af None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:49.671398 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:49.678665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4cddd-183e-4d40-8abd-bd39b724ee0b None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:49.679152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4cddd-183e-4d40-8abd-bd39b724ee0b None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:49.679259 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4cddd-183e-4d40-8abd-bd39b724ee0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:49.737233 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:49.759425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-ebe4cddd-183e-4d40-8abd-bd39b724ee0b None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:49.760665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ebe4cddd-183e-4d40-8abd-bd39b724ee0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:49.859029 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:50.010991 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95913ec3-5891-4437-92dc-7e13a31d12af None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:50.012588 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-95913ec3-5891-4437-92dc-7e13a31d12af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:50.161110 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95913ec3-5891-4437-92dc-7e13a31d12af None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:50.173056 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:50.280133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558375b-a8a8-4b1f-afae-e3adc75e3596 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:50.280790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558375b-a8a8-4b1f-afae-e3adc75e3596 service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:50.281074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558375b-a8a8-4b1f-afae-e3adc75e3596 service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:50.281353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558375b-a8a8-4b1f-afae-e3adc75e3596 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:50.283036 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9558375b-a8a8-4b1f-afae-e3adc75e3596 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:50.374553 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558375b-a8a8-4b1f-afae-e3adc75e3596 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:50.376161 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558375b-a8a8-4b1f-afae-e3adc75e3596 service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:50.394421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 718/1422] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:51.262866 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209bddc-018b-49c1-b821-398737eaf682 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:51.263441 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209bddc-018b-49c1-b821-398737eaf682 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:51.263441 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209bddc-018b-49c1-b821-398737eaf682 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:51.264831 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:51.269930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c68461-3cc5-4dad-bc80-8d074944ae0a None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:51.270375 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c68461-3cc5-4dad-bc80-8d074944ae0a None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:51.270523 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c68461-3cc5-4dad-bc80-8d074944ae0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:51.290142 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:51.346580 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-04c68461-3cc5-4dad-bc80-8d074944ae0a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:51.347718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-04c68461-3cc5-4dad-bc80-8d074944ae0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:51.452427 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:51.605459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209bddc-018b-49c1-b821-398737eaf682 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:51.607311 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-6209bddc-018b-49c1-b821-398737eaf682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:51.724915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:51.766067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209bddc-018b-49c1-b821-398737eaf682 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:51.885834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0a004-31b2-4572-b758-36169120518b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:51.886545 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0a004-31b2-4572-b758-36169120518b service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:51.886831 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0a004-31b2-4572-b758-36169120518b service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:51.887104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0a004-31b2-4572-b758-36169120518b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:51.888808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-66d0a004-31b2-4572-b758-36169120518b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:51.979042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:51.993934 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0a004-31b2-4572-b758-36169120518b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:51.996780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0a004-31b2-4572-b758-36169120518b service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:52.021320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 706/1425] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:52.906917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1edf0-2c58-41a9-aad1-163a8bb3120c None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:52.907483 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1edf0-2c58-41a9-aad1-163a8bb3120c None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:52.907483 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1edf0-2c58-41a9-aad1-163a8bb3120c None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:52.908763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:52.916754 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12045f4e-9a4f-47dc-b12a-df18363bf21d None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:52.917220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12045f4e-9a4f-47dc-b12a-df18363bf21d None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:52.917326 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12045f4e-9a4f-47dc-b12a-df18363bf21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:52.993733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-12045f4e-9a4f-47dc-b12a-df18363bf21d None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:52.994980 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-12045f4e-9a4f-47dc-b12a-df18363bf21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:53.092720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:53.246664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1edf0-2c58-41a9-aad1-163a8bb3120c None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:53.248089 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d8b1edf0-2c58-41a9-aad1-163a8bb3120c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:53.334842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:53.397334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1edf0-2c58-41a9-aad1-163a8bb3120c None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:53.527742 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6c10f-43c1-4e11-8a54-514c7e74d3ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:53.527742 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6c10f-43c1-4e11-8a54-514c7e74d3ea service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:53.527742 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6c10f-43c1-4e11-8a54-514c7e74d3ea service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:53.527742 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6c10f-43c1-4e11-8a54-514c7e74d3ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:53.528675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-3fa6c10f-43c1-4e11-8a54-514c7e74d3ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:53.576509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:53.624854 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6c10f-43c1-4e11-8a54-514c7e74d3ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:53.626602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6c10f-43c1-4e11-8a54-514c7e74d3ea service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:53.653582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 721/1428] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:54.562996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a616c8f-f83e-4244-8127-e82adced61ad None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:54.563675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a616c8f-f83e-4244-8127-e82adced61ad None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:54.563675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a616c8f-f83e-4244-8127-e82adced61ad None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:54.564975 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:24:54.570662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79605dc-6e46-4e35-9222-dfa8c27f0a8a None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:54.571120 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79605dc-6e46-4e35-9222-dfa8c27f0a8a None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:54.571221 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79605dc-6e46-4e35-9222-dfa8c27f0a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:54.649764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-b79605dc-6e46-4e35-9222-dfa8c27f0a8a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:54.651037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b79605dc-6e46-4e35-9222-dfa8c27f0a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:54.756274 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:54.906499 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a616c8f-f83e-4244-8127-e82adced61ad None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:54.908023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8a616c8f-f83e-4244-8127-e82adced61ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:54.960958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:55.063870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a616c8f-f83e-4244-8127-e82adced61ad None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:55.182389 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98acef-aeb5-496c-babc-212c37b277af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:55.182964 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98acef-aeb5-496c-babc-212c37b277af service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:55.183247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98acef-aeb5-496c-babc-212c37b277af service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:55.183519 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98acef-aeb5-496c-babc-212c37b277af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:55.185163 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0c98acef-aeb5-496c-babc-212c37b277af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:55.192138 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:55.276841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c98acef-aeb5-496c-babc-212c37b277af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:55.278469 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c98acef-aeb5-496c-babc-212c37b277af service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:55.297018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 709/1431] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:56.180270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc32b6d-36ee-47ac-b18a-551041651ead None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:56.180843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc32b6d-36ee-47ac-b18a-551041651ead None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:56.180962 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc32b6d-36ee-47ac-b18a-551041651ead None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:56.182609 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:24:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:56.187747 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3670def-01a2-445f-9a3e-52c1c4868c96 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:56.188185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3670def-01a2-445f-9a3e-52c1c4868c96 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:56.188287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3670def-01a2-445f-9a3e-52c1c4868c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:56.263454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-d3670def-01a2-445f-9a3e-52c1c4868c96 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:56.264679 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d3670def-01a2-445f-9a3e-52c1c4868c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:56.333461 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:56.364676 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:56.512037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc32b6d-36ee-47ac-b18a-551041651ead None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:56.513545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9dc32b6d-36ee-47ac-b18a-551041651ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:56.532300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:56.662934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc32b6d-36ee-47ac-b18a-551041651ead None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:56.760933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:56.788800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc2cb5-9233-4c3d-8e2a-91db87f01dfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:56.789450 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc2cb5-9233-4c3d-8e2a-91db87f01dfd service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:56.789735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc2cb5-9233-4c3d-8e2a-91db87f01dfd service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:56.790015 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc2cb5-9233-4c3d-8e2a-91db87f01dfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:56.791692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-11dc2cb5-9233-4c3d-8e2a-91db87f01dfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:56.885794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc2cb5-9233-4c3d-8e2a-91db87f01dfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:56.887510 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc2cb5-9233-4c3d-8e2a-91db87f01dfd service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:56.907690 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 724/1434] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:57.794693 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd27f4-682a-46c1-a995-d8749ab40588 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:57.794693 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd27f4-682a-46c1-a995-d8749ab40588 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:57.794693 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd27f4-682a-46c1-a995-d8749ab40588 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:57.795874 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:24:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:57.800498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4303dd5-619d-46fb-9c68-a27cac81c5c6 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:57.801013 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4303dd5-619d-46fb-9c68-a27cac81c5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:57.801123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4303dd5-619d-46fb-9c68-a27cac81c5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:57.886089 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-d4303dd5-619d-46fb-9c68-a27cac81c5c6 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:57.887350 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d4303dd5-619d-46fb-9c68-a27cac81c5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:57.931423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:57.999850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:24:58.151190 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd27f4-682a-46c1-a995-d8749ab40588 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:58.152861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-c2bd27f4-682a-46c1-a995-d8749ab40588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:58.316857 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd27f4-682a-46c1-a995-d8749ab40588 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:24:58.372839 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:58.433124 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67967f8-3820-430d-a472-efe6f2bec279 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:24:58.433785 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67967f8-3820-430d-a472-efe6f2bec279 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:58.434072 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67967f8-3820-430d-a472-efe6f2bec279 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:58.434375 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67967f8-3820-430d-a472-efe6f2bec279 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:58.436038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e67967f8-3820-430d-a472-efe6f2bec279 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:58.548451 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67967f8-3820-430d-a472-efe6f2bec279 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:24:58.550050 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67967f8-3820-430d-a472-efe6f2bec279 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:24:58.569183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 712/1437] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:24:59.561084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b367b74-05e2-44c6-9bb3-6f2c0f4b8cbf None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:59.561735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b367b74-05e2-44c6-9bb3-6f2c0f4b8cbf None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:59.561735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b367b74-05e2-44c6-9bb3-6f2c0f4b8cbf None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:59.563131 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:24:59.568419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e460aca-657c-4310-9fd5-400831d30e0f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:24:59.568880 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e460aca-657c-4310-9fd5-400831d30e0f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:24:59.569145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e460aca-657c-4310-9fd5-400831d30e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:24:59.652455 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-4e460aca-657c-4310-9fd5-400831d30e0f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:24:59.653688 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4e460aca-657c-4310-9fd5-400831d30e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:24:59.655904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:59.761967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:24:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:24:59.917757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b367b74-05e2-44c6-9bb3-6f2c0f4b8cbf None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:24:59.919253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9b367b74-05e2-44c6-9bb3-6f2c0f4b8cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:00.099455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b367b74-05e2-44c6-9bb3-6f2c0f4b8cbf None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:00.131708 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:00.229280 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb191cf-d887-441e-a8ef-2293b1d971dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:00.229844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb191cf-d887-441e-a8ef-2293b1d971dd service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:00.230131 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb191cf-d887-441e-a8ef-2293b1d971dd service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:00.230444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb191cf-d887-441e-a8ef-2293b1d971dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:00.232419 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-cfb191cf-d887-441e-a8ef-2293b1d971dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:00.330155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb191cf-d887-441e-a8ef-2293b1d971dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:00.331860 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb191cf-d887-441e-a8ef-2293b1d971dd service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:00.352286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 727/1440] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:24:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:02.312818 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ed921-3b1a-4854-b2f4-876bd50c04ae None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:02.313674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ed921-3b1a-4854-b2f4-876bd50c04ae None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:02.313674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ed921-3b1a-4854-b2f4-876bd50c04ae None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:02.314849 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:02.321122 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cbea3-fdb6-43a4-8ba7-980eaa4a81e9 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:02.321620 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cbea3-fdb6-43a4-8ba7-980eaa4a81e9 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:02.321781 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cbea3-fdb6-43a4-8ba7-980eaa4a81e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:02.369885 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:02.403601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-155cbea3-fdb6-43a4-8ba7-980eaa4a81e9 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:02.404904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-155cbea3-fdb6-43a4-8ba7-980eaa4a81e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:02.529172 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:02.569781 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ed921-3b1a-4854-b2f4-876bd50c04ae None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:02.571374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d78ed921-3b1a-4854-b2f4-876bd50c04ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:02.690095 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ed921-3b1a-4854-b2f4-876bd50c04ae None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:02.723983 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:02.765520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1051a8c-1ba4-4ec8-af63-e0de5326cbc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:02.766202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1051a8c-1ba4-4ec8-af63-e0de5326cbc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:02.766534 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1051a8c-1ba4-4ec8-af63-e0de5326cbc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:02.766811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1051a8c-1ba4-4ec8-af63-e0de5326cbc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:02.768441 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b1051a8c-1ba4-4ec8-af63-e0de5326cbc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:02.879354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1051a8c-1ba4-4ec8-af63-e0de5326cbc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:02.880868 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1051a8c-1ba4-4ec8-af63-e0de5326cbc9 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:02.909201 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 715/1443] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:05.159910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbc653-65a0-4dfe-97f0-2cc201e575aa None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:05.160561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbc653-65a0-4dfe-97f0-2cc201e575aa None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:05.160561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbc653-65a0-4dfe-97f0-2cc201e575aa None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:05.162709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:25:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:05.168444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898c592-5748-48fd-9641-9ca13cf9c61f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:05.168882 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898c592-5748-48fd-9641-9ca13cf9c61f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:05.169002 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898c592-5748-48fd-9641-9ca13cf9c61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:05.238694 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-6898c592-5748-48fd-9641-9ca13cf9c61f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:05.240412 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-6898c592-5748-48fd-9641-9ca13cf9c61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:05.253999 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:05.344113 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:05.384083 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbc653-65a0-4dfe-97f0-2cc201e575aa None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:05.385611 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-39cbc653-65a0-4dfe-97f0-2cc201e575aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:05.491052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbc653-65a0-4dfe-97f0-2cc201e575aa None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:05.503268 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:05.566093 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4a8f7-20f6-4141-8328-c0efc02eb22f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:05.566774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4a8f7-20f6-4141-8328-c0efc02eb22f service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:05.567056 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4a8f7-20f6-4141-8328-c0efc02eb22f service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:05.567356 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4a8f7-20f6-4141-8328-c0efc02eb22f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:05.568994 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9fc4a8f7-20f6-4141-8328-c0efc02eb22f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:05.681965 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4a8f7-20f6-4141-8328-c0efc02eb22f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:05.683707 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4a8f7-20f6-4141-8328-c0efc02eb22f service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:05.703227 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 730/1446] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:25:08.281390 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5f0fa-6990-4fe7-ac96-2a6e3cd0b694 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:08.281964 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5f0fa-6990-4fe7-ac96-2a6e3cd0b694 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:08.281964 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5f0fa-6990-4fe7-ac96-2a6e3cd0b694 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:08.283377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:08.289378 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f0fb8-2cb9-440b-89ed-837758a470a2 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:08.289860 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f0fb8-2cb9-440b-89ed-837758a470a2 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:08.290001 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f0fb8-2cb9-440b-89ed-837758a470a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:08.341369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:08.362377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-363f0fb8-2cb9-440b-89ed-837758a470a2 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:08.362377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-363f0fb8-2cb9-440b-89ed-837758a470a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:08.476462 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:08.515473 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5f0fa-6990-4fe7-ac96-2a6e3cd0b694 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:08.517023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-89d5f0fa-6990-4fe7-ac96-2a6e3cd0b694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:08.626956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5f0fa-6990-4fe7-ac96-2a6e3cd0b694 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:08.668912 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:08.713417 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1445a-7a86-4283-8f6e-2b5cc32e86b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:08.714212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1445a-7a86-4283-8f6e-2b5cc32e86b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:08.714556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1445a-7a86-4283-8f6e-2b5cc32e86b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:08.714836 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1445a-7a86-4283-8f6e-2b5cc32e86b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:08.716612 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-3ea1445a-7a86-4283-8f6e-2b5cc32e86b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:08.849447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1445a-7a86-4283-8f6e-2b5cc32e86b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:08.851116 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1445a-7a86-4283-8f6e-2b5cc32e86b5 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:08.870673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 718/1449] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:11.044850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49bc36-e895-48f4-9164-c326deb3920f None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:11.045642 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49bc36-e895-48f4-9164-c326deb3920f None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:11.045642 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49bc36-e895-48f4-9164-c326deb3920f None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:11.046862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:25:11.053073 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8286c28-d30e-4a84-91e7-148d13b7be19 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:11.053631 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8286c28-d30e-4a84-91e7-148d13b7be19 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:11.053631 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8286c28-d30e-4a84-91e7-148d13b7be19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:11.124612 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-b8286c28-d30e-4a84-91e7-148d13b7be19 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:11.125827 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b8286c28-d30e-4a84-91e7-148d13b7be19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:11.146728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:11.223929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:11.263541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49bc36-e895-48f4-9164-c326deb3920f None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:11.265099 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9d49bc36-e895-48f4-9164-c326deb3920f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:11.376897 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49bc36-e895-48f4-9164-c326deb3920f None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:11.399319 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:11.459965 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c18fd3-b533-4211-964a-397b0b411236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:11.460624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c18fd3-b533-4211-964a-397b0b411236 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:11.460909 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c18fd3-b533-4211-964a-397b0b411236 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:11.461187 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c18fd3-b533-4211-964a-397b0b411236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:11.462885 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-42c18fd3-b533-4211-964a-397b0b411236 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:11.602824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c18fd3-b533-4211-964a-397b0b411236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:11.604326 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c18fd3-b533-4211-964a-397b0b411236 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:11.624552 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 733/1452] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:13.772745 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40ea3c-8d21-452a-9833-132f22f63eaa None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:13.773351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40ea3c-8d21-452a-9833-132f22f63eaa None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:13.773452 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40ea3c-8d21-452a-9833-132f22f63eaa None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:13.775105 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:13.782934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d292f-9452-444d-8290-e7841caa357e None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:13.783771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d292f-9452-444d-8290-e7841caa357e None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:13.783947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d292f-9452-444d-8290-e7841caa357e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:13.850559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:13.853566 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-957d292f-9452-444d-8290-e7841caa357e None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:13.856046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-957d292f-9452-444d-8290-e7841caa357e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:13.971210 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:14.010868 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40ea3c-8d21-452a-9833-132f22f63eaa None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:14.012442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-1a40ea3c-8d21-452a-9833-132f22f63eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:14.142662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40ea3c-8d21-452a-9833-132f22f63eaa None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:14.209414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:14.235210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619b382-d431-4e91-9645-d26169d2d7b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:14.235867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619b382-d431-4e91-9645-d26169d2d7b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:14.236144 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619b382-d431-4e91-9645-d26169d2d7b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:14.236418 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619b382-d431-4e91-9645-d26169d2d7b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:14.238113 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b619b382-d431-4e91-9645-d26169d2d7b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:14.359027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619b382-d431-4e91-9645-d26169d2d7b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:14.360375 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619b382-d431-4e91-9645-d26169d2d7b6 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:14.380130 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 721/1455] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:25:16.647130 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a1d8a-bd87-476b-ac32-70f964e4876d None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:16.649224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a1d8a-bd87-476b-ac32-70f964e4876d None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:16.649224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a1d8a-bd87-476b-ac32-70f964e4876d None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:16.649224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:16.657659 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad039e-6e9b-4f16-be4c-c05520138bdf None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:16.662388 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad039e-6e9b-4f16-be4c-c05520138bdf None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:16.662388 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad039e-6e9b-4f16-be4c-c05520138bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:16.736003 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-32ad039e-6e9b-4f16-be4c-c05520138bdf None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:16.737422 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-32ad039e-6e9b-4f16-be4c-c05520138bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:16.771942 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:16.853810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:16.921557 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a1d8a-bd87-476b-ac32-70f964e4876d None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:16.924349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-448a1d8a-bd87-476b-ac32-70f964e4876d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:17.109193 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a1d8a-bd87-476b-ac32-70f964e4876d None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:17.159103 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:17.220171 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcb79d-693e-4de3-adec-572486bc1bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:17.220970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcb79d-693e-4de3-adec-572486bc1bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:17.221379 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcb79d-693e-4de3-adec-572486bc1bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:17.221702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcb79d-693e-4de3-adec-572486bc1bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:17.224795 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c0dcb79d-693e-4de3-adec-572486bc1bc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:17.345873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcb79d-693e-4de3-adec-572486bc1bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:17.347560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcb79d-693e-4de3-adec-572486bc1bc6 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:17.368365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 736/1458] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:19.522549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c5442-de3b-4625-943c-252c3ffd920a None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:19.523110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c5442-de3b-4625-943c-252c3ffd920a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:19.523110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c5442-de3b-4625-943c-252c3ffd920a None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:19.524580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:19.529857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f672f4-692f-410f-871e-9b4306d6f41b None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:19.530767 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f672f4-692f-410f-871e-9b4306d6f41b None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:19.531071 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f672f4-692f-410f-871e-9b4306d6f41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:19.597490 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-d1f672f4-692f-410f-871e-9b4306d6f41b None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:19.599271 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d1f672f4-692f-410f-871e-9b4306d6f41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:19.608128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:19.696664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:19.735427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c5442-de3b-4625-943c-252c3ffd920a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:19.737037 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-862c5442-de3b-4625-943c-252c3ffd920a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:19.852948 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c5442-de3b-4625-943c-252c3ffd920a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:19.914122 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:19.947197 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4d59a-5236-41ac-a40c-c6b326d2eb7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:19.948327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4d59a-5236-41ac-a40c-c6b326d2eb7c service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:19.948819 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4d59a-5236-41ac-a40c-c6b326d2eb7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:19.949290 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4d59a-5236-41ac-a40c-c6b326d2eb7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:19.952374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-25b4d59a-5236-41ac-a40c-c6b326d2eb7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:20.091303 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4d59a-5236-41ac-a40c-c6b326d2eb7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:20.093090 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4d59a-5236-41ac-a40c-c6b326d2eb7c service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:20.115804 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 724/1461] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:22.257774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9ce21-bffc-4ab9-a257-9a36c12f1d08 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:22.258349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9ce21-bffc-4ab9-a257-9a36c12f1d08 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:22.258452 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9ce21-bffc-4ab9-a257-9a36c12f1d08 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:22.259759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:25:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:22.265704 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abc6d1-f088-448e-83b3-7fc0c6db0b95 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:22.266110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abc6d1-f088-448e-83b3-7fc0c6db0b95 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:22.266253 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abc6d1-f088-448e-83b3-7fc0c6db0b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:22.336306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-b6abc6d1-f088-448e-83b3-7fc0c6db0b95 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:22.337531 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b6abc6d1-f088-448e-83b3-7fc0c6db0b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:22.378064 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:22.439820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:22.478352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9ce21-bffc-4ab9-a257-9a36c12f1d08 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:22.480057 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b0b9ce21-bffc-4ab9-a257-9a36c12f1d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:22.585491 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9ce21-bffc-4ab9-a257-9a36c12f1d08 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:22.628616 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:22.660932 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50200304-17e5-4171-9e0b-2f89f8d36fa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:22.661579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50200304-17e5-4171-9e0b-2f89f8d36fa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:22.661857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50200304-17e5-4171-9e0b-2f89f8d36fa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:22.662127 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50200304-17e5-4171-9e0b-2f89f8d36fa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:22.663797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-50200304-17e5-4171-9e0b-2f89f8d36fa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:22.775696 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50200304-17e5-4171-9e0b-2f89f8d36fa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:22.777316 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50200304-17e5-4171-9e0b-2f89f8d36fa9 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:22.796739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 739/1464] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:24.848637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad230bb5-22ac-4cf3-9270-68b009f66852 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:24.849534 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad230bb5-22ac-4cf3-9270-68b009f66852 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:24.849534 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad230bb5-22ac-4cf3-9270-68b009f66852 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:24.850776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:24.856688 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a79ec-0d3d-4024-809f-384af3074c8c None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:24.857434 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a79ec-0d3d-4024-809f-384af3074c8c None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:24.857434 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a79ec-0d3d-4024-809f-384af3074c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:24.935049 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-ae0a79ec-0d3d-4024-809f-384af3074c8c None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:24.936617 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ae0a79ec-0d3d-4024-809f-384af3074c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:24.957462 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:25.060201 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:25.100600 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad230bb5-22ac-4cf3-9270-68b009f66852 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:25.102214 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ad230bb5-22ac-4cf3-9270-68b009f66852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:25.219627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad230bb5-22ac-4cf3-9270-68b009f66852 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:25.294397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:25.303454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82884765-cddb-434b-ac22-ad8e1dfff0eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:25.307584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82884765-cddb-434b-ac22-ad8e1dfff0eb service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:25.307584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82884765-cddb-434b-ac22-ad8e1dfff0eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:25.307584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82884765-cddb-434b-ac22-ad8e1dfff0eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:25.307584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-82884765-cddb-434b-ac22-ad8e1dfff0eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:25.431757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82884765-cddb-434b-ac22-ad8e1dfff0eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:25.433198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82884765-cddb-434b-ac22-ad8e1dfff0eb service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:25.456289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 727/1467] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:25:27.596778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ee4df-5b6a-46d3-9ef5-3a29f5e7db23 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:27.597379 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ee4df-5b6a-46d3-9ef5-3a29f5e7db23 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:27.597379 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ee4df-5b6a-46d3-9ef5-3a29f5e7db23 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:27.598845 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:25:27.604443 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d6932-570d-49e6-a7f4-4e0bc17045dc None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:27.604958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d6932-570d-49e6-a7f4-4e0bc17045dc None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:27.605390 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d6932-570d-49e6-a7f4-4e0bc17045dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:27.675592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-180d6932-570d-49e6-a7f4-4e0bc17045dc None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:27.676857 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-180d6932-570d-49e6-a7f4-4e0bc17045dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:27.727895 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:27.786333 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:25:27.824109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ee4df-5b6a-46d3-9ef5-3a29f5e7db23 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:27.825650 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2e0ee4df-5b6a-46d3-9ef5-3a29f5e7db23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:27.935868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ee4df-5b6a-46d3-9ef5-3a29f5e7db23 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:27.986625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:28.015405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54367112-87cb-4305-9639-f13cf7bbf6ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:28.016061 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54367112-87cb-4305-9639-f13cf7bbf6ce service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:28.016341 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54367112-87cb-4305-9639-f13cf7bbf6ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:28.016613 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54367112-87cb-4305-9639-f13cf7bbf6ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:28.018356 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-54367112-87cb-4305-9639-f13cf7bbf6ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:28.150155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54367112-87cb-4305-9639-f13cf7bbf6ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:28.151649 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54367112-87cb-4305-9639-f13cf7bbf6ce service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:28.171432 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 742/1470] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:37.789888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=22 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:38.168673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=66 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:48.749126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f63478-4482-41b5-8cfe-50a2e476cccc None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:48.749711 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f63478-4482-41b5-8cfe-50a2e476cccc None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:48.749711 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f63478-4482-41b5-8cfe-50a2e476cccc None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:48.751258 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 729/1471] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:25:48.763279 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a5dcb-6b36-473d-9b4c-1de59890c7b2 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:48.763963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a5dcb-6b36-473d-9b4c-1de59890c7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:48.763963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a5dcb-6b36-473d-9b4c-1de59890c7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:48.834140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-608a5dcb-6b36-473d-9b4c-1de59890c7b2 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:48.835406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-608a5dcb-6b36-473d-9b4c-1de59890c7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:48.941439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 743/1472] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:49.030887 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66756a8c-290e-4421-a8c5-b89b1cbd1038 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:49.031212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66756a8c-290e-4421-a8c5-b89b1cbd1038 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:49.031527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66756a8c-290e-4421-a8c5-b89b1cbd1038 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:49.032938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 730/1473] 10.222.0.78 () {54 vars in 966 bytes} [Wed Jul 20 15:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:49.038344 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad1f2d-362e-49dc-8ce0-01c641fcdc86 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:49.038642 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad1f2d-362e-49dc-8ce0-01c641fcdc86 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:49.038924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad1f2d-362e-49dc-8ce0-01c641fcdc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:49.120826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-17ad1f2d-362e-49dc-8ce0-01c641fcdc86 None None] MFA Rules not processed for user `b6c13596c2df438294bdac7c52ae8cb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:49.122053 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-17ad1f2d-362e-49dc-8ce0-01c641fcdc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:49.143131 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:49.211402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 744/1474] 10.222.0.78 () {58 vars in 1063 bytes} [Wed Jul 20 15:25:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:49.219172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66756a8c-290e-4421-a8c5-b89b1cbd1038 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:49.220695 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-66756a8c-290e-4421-a8c5-b89b1cbd1038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:49.336045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66756a8c-290e-4421-a8c5-b89b1cbd1038 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:49.412420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a37cef-0481-4dc1-81f0-6a84eb8ad8ec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6c13596c2df438294bdac7c52ae8cb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:49.413061 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a37cef-0481-4dc1-81f0-6a84eb8ad8ec service glance] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:49.413347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a37cef-0481-4dc1-81f0-6a84eb8ad8ec service glance] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:49.413619 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a37cef-0481-4dc1-81f0-6a84eb8ad8ec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:49.415320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-36a37cef-0481-4dc1-81f0-6a84eb8ad8ec service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:49.470373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:49.532500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a37cef-0481-4dc1-81f0-6a84eb8ad8ec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:49.533937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a37cef-0481-4dc1-81f0-6a84eb8ad8ec service glance] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:49.560533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 731/1475] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:51.776006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c1c6e-5222-4c3b-801d-f0fa42c3ac48 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:51.776570 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c1c6e-5222-4c3b-801d-f0fa42c3ac48 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:51.776570 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c1c6e-5222-4c3b-801d-f0fa42c3ac48 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:51.778007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 745/1476] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:51.785473 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9248d-ff6c-445f-bd7d-ea99afdc42ea None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:51.791407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9248d-ff6c-445f-bd7d-ea99afdc42ea None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:51.791407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9248d-ff6c-445f-bd7d-ea99afdc42ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:51.861847 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-02b9248d-ff6c-445f-bd7d-ea99afdc42ea None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:51.863852 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-02b9248d-ff6c-445f-bd7d-ea99afdc42ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:51.964825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 732/1477] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:25:51.990002 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c1c6e-5222-4c3b-801d-f0fa42c3ac48 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:51.991558 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-4a9c1c6e-5222-4c3b-801d-f0fa42c3ac48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:52.120037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c1c6e-5222-4c3b-801d-f0fa42c3ac48 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:52.170488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:52.216637 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449c59e-3013-4ae4-94e8-2506f2983bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:52.217299 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449c59e-3013-4ae4-94e8-2506f2983bfb admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:52.217578 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449c59e-3013-4ae4-94e8-2506f2983bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:52.217847 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449c59e-3013-4ae4-94e8-2506f2983bfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:52.218607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449c59e-3013-4ae4-94e8-2506f2983bfb admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:52.220431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: WARNING py.warnings [None req-a449c59e-3013-4ae4-94e8-2506f2983bfb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:25:52.220431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: warnings.warn(msg) Jul 20 15:25:52.220431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: Jul 20 15:25:52.221693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449c59e-3013-4ae4-94e8-2506f2983bfb admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:52.243123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 746/1478] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Jul 20 15:25:51 2022] GET /identity/v3/projects => generated 2553 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:25:54.379208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbcacb-cbfd-4bd9-af8e-0b21a3c03e54 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:54.379780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbcacb-cbfd-4bd9-af8e-0b21a3c03e54 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:54.379780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbcacb-cbfd-4bd9-af8e-0b21a3c03e54 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:54.381057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 733/1479] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:25:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:54.386518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb1070-6dcf-45ee-9483-f0d5caa44a41 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:54.386913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb1070-6dcf-45ee-9483-f0d5caa44a41 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:54.387069 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb1070-6dcf-45ee-9483-f0d5caa44a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:54.461408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6dcb1070-6dcf-45ee-9483-f0d5caa44a41 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:54.462766 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6dcb1070-6dcf-45ee-9483-f0d5caa44a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:54.562209 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 747/1480] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:25:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:54.606941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbcacb-cbfd-4bd9-af8e-0b21a3c03e54 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:54.608536 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8fcbcacb-cbfd-4bd9-af8e-0b21a3c03e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:54.619066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:54.772185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbcacb-cbfd-4bd9-af8e-0b21a3c03e54 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:54.864624 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:54.898924 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682fc77-75fc-4b49-8198-fa2a544a30c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:54.899583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682fc77-75fc-4b49-8198-fa2a544a30c6 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:54.899870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682fc77-75fc-4b49-8198-fa2a544a30c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:54.900156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682fc77-75fc-4b49-8198-fa2a544a30c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:54.902043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-6682fc77-75fc-4b49-8198-fa2a544a30c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:55.010827 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682fc77-75fc-4b49-8198-fa2a544a30c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:55.010827 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682fc77-75fc-4b49-8198-fa2a544a30c6 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:55.028767 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 734/1481] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:25:57.342755 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6429d-561c-4284-8e23-b8aade015197 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:57.343371 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6429d-561c-4284-8e23-b8aade015197 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:57.343371 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6429d-561c-4284-8e23-b8aade015197 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:57.344714 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 748/1482] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:25:57.350680 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802d01d-620e-4407-abb4-df3d7ccb7ac7 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:57.351096 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802d01d-620e-4407-abb4-df3d7ccb7ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:57.351232 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802d01d-620e-4407-abb4-df3d7ccb7ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:57.417748 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-f802d01d-620e-4407-abb4-df3d7ccb7ac7 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:25:57.419009 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f802d01d-620e-4407-abb4-df3d7ccb7ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:57.470810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:57.523364 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 735/1483] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:25:57.560512 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6429d-561c-4284-8e23-b8aade015197 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:25:57.562046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-beb6429d-561c-4284-8e23-b8aade015197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:57.587804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:57.667210 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6429d-561c-4284-8e23-b8aade015197 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:25:57.743995 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b622126-d5c8-455f-b3d4-019a1a106ffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:25:57.744638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b622126-d5c8-455f-b3d4-019a1a106ffe service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:25:57.744917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b622126-d5c8-455f-b3d4-019a1a106ffe service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:25:57.745190 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b622126-d5c8-455f-b3d4-019a1a106ffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:25:57.746881 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9b622126-d5c8-455f-b3d4-019a1a106ffe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:25:57.824560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:57.859395 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b622126-d5c8-455f-b3d4-019a1a106ffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:25:57.860826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b622126-d5c8-455f-b3d4-019a1a106ffe service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:25:57.880954 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 749/1484] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:25:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:26:00.009701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1081b9f-2730-4c2e-80ea-d9f2cdfb9668 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:00.010212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1081b9f-2730-4c2e-80ea-d9f2cdfb9668 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:00.010212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1081b9f-2730-4c2e-80ea-d9f2cdfb9668 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:00.011664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 736/1485] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:26:00.016922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c1e6a-3446-4730-a9ab-cbaaead303df None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:00.017394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c1e6a-3446-4730-a9ab-cbaaead303df None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:00.017522 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c1e6a-3446-4730-a9ab-cbaaead303df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:00.083635 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-626c1e6a-3446-4730-a9ab-cbaaead303df None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:00.084838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-626c1e6a-3446-4730-a9ab-cbaaead303df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:00.186465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 750/1486] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:00.224877 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1081b9f-2730-4c2e-80ea-d9f2cdfb9668 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:00.226447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a1081b9f-2730-4c2e-80ea-d9f2cdfb9668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:00.344537 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1081b9f-2730-4c2e-80ea-d9f2cdfb9668 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:00.423405 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:00.425687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4ba51-deb2-4423-a5f2-e7ac1a06baa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:00.426371 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4ba51-deb2-4423-a5f2-e7ac1a06baa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:00.426655 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4ba51-deb2-4423-a5f2-e7ac1a06baa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:00.426941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4ba51-deb2-4423-a5f2-e7ac1a06baa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:00.428664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-11e4ba51-deb2-4423-a5f2-e7ac1a06baa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:00.540830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4ba51-deb2-4423-a5f2-e7ac1a06baa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:00.542276 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4ba51-deb2-4423-a5f2-e7ac1a06baa7 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:00.565089 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 737/1487] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:26:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:26:02.803427 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed1998-7df5-465c-a97b-4025951447ef None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:02.803959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed1998-7df5-465c-a97b-4025951447ef None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:02.803959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed1998-7df5-465c-a97b-4025951447ef None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:02.805345 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 751/1488] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:26:02.810580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2dba2-18ca-4444-812f-25a951c5672f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:02.811022 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2dba2-18ca-4444-812f-25a951c5672f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:02.811163 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2dba2-18ca-4444-812f-25a951c5672f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:02.875979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-74c2dba2-18ca-4444-812f-25a951c5672f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:02.877218 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-74c2dba2-18ca-4444-812f-25a951c5672f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:02.929017 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:02.973772 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 738/1489] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:03.015512 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed1998-7df5-465c-a97b-4025951447ef None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:03.017126 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-80ed1998-7df5-465c-a97b-4025951447ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:03.054597 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:03.129475 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed1998-7df5-465c-a97b-4025951447ef None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:03.207711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0d2ef-bd68-44dd-9bc8-064a4bb362d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:03.207711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0d2ef-bd68-44dd-9bc8-064a4bb362d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:03.208473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0d2ef-bd68-44dd-9bc8-064a4bb362d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:03.208473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0d2ef-bd68-44dd-9bc8-064a4bb362d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:03.209906 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-5fd0d2ef-bd68-44dd-9bc8-064a4bb362d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:03.301584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:03.331538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0d2ef-bd68-44dd-9bc8-064a4bb362d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:03.332981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0d2ef-bd68-44dd-9bc8-064a4bb362d0 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:03.353388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 752/1490] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:26:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:26:05.948470 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d4289-d2b0-473a-8b1e-d608ae7b7185 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:05.949048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d4289-d2b0-473a-8b1e-d608ae7b7185 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:05.949048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d4289-d2b0-473a-8b1e-d608ae7b7185 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:05.950459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 739/1491] 10.222.0.78 () {54 vars in 926 bytes} [Wed Jul 20 15:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:26:05.956115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa10ab9-7cfe-464d-8064-bfda63053050 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:05.956618 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa10ab9-7cfe-464d-8064-bfda63053050 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:05.956750 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa10ab9-7cfe-464d-8064-bfda63053050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:06.022822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-5fa10ab9-7cfe-464d-8064-bfda63053050 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:26:06.024034 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-5fa10ab9-7cfe-464d-8064-bfda63053050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:06.123479 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 753/1492] 10.222.0.78 () {58 vars in 1023 bytes} [Wed Jul 20 15:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:26:06.163282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d4289-d2b0-473a-8b1e-d608ae7b7185 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:26:06.164849 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-453d4289-d2b0-473a-8b1e-d608ae7b7185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:06.272950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d4289-d2b0-473a-8b1e-d608ae7b7185 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:26:06.347155 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2b7b5-ba5c-4316-8d9e-65be8b25496b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:26:06.347802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2b7b5-ba5c-4316-8d9e-65be8b25496b service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:26:06.348083 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2b7b5-ba5c-4316-8d9e-65be8b25496b service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:26:06.348355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2b7b5-ba5c-4316-8d9e-65be8b25496b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:26:06.349990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-6db2b7b5-ba5c-4316-8d9e-65be8b25496b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:26:06.359798 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:06.469981 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2b7b5-ba5c-4316-8d9e-65be8b25496b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:26:06.472149 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2b7b5-ba5c-4316-8d9e-65be8b25496b service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:26:06.492335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 740/1493] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:26:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:26:16.121664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=82 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:16.489916 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=50 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:47.272941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c36f6-9d81-4f9f-b2e5-0e438abb0b22 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:47.273541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c36f6-9d81-4f9f-b2e5-0e438abb0b22 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:47.273541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c36f6-9d81-4f9f-b2e5-0e438abb0b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:47.355559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-a09c36f6-9d81-4f9f-b2e5-0e438abb0b22 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:27:47.356709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-a09c36f6-9d81-4f9f-b2e5-0e438abb0b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:47.474198 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 754/1494] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:27:47.483620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37534164-a674-4669-a896-1ba9b0551bd4 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:47.484272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37534164-a674-4669-a896-1ba9b0551bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:47.484450 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37534164-a674-4669-a896-1ba9b0551bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:47.574176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-37534164-a674-4669-a896-1ba9b0551bd4 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:27:47.575495 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-37534164-a674-4669-a896-1ba9b0551bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:47.690521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 741/1495] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:27:47.695477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c36f6-9d81-4f9f-b2e5-0e438abb0b22 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:47.696987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-a09c36f6-9d81-4f9f-b2e5-0e438abb0b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:47.710443 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c36f6-9d81-4f9f-b2e5-0e438abb0b22 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:47.779597 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:47.841321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c3806-90b4-45ed-b3b8-7c0c01a9aa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:47.842522 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c3806-90b4-45ed-b3b8-7c0c01a9aa92 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:47.843013 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c3806-90b4-45ed-b3b8-7c0c01a9aa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:47.843538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c3806-90b4-45ed-b3b8-7c0c01a9aa92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:47.844890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c3806-90b4-45ed-b3b8-7c0c01a9aa92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:27:47.845438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c3806-90b4-45ed-b3b8-7c0c01a9aa92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:47.847366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: WARNING py.warnings [None req-617c3806-90b4-45ed-b3b8-7c0c01a9aa92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:27:47.847366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: warnings.warn(msg) Jul 20 15:27:47.847366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: Jul 20 15:27:47.849088 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c3806-90b4-45ed-b3b8-7c0c01a9aa92 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:47.870697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 755/1496] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:27:47.879690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37534164-a674-4669-a896-1ba9b0551bd4 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:47.889745 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37534164-a674-4669-a896-1ba9b0551bd4 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:47.964258 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:48.002140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23bbcc1-5bfa-403c-a41b-590d077aa800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:48.003021 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bbcc1-5bfa-403c-a41b-590d077aa800 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:48.003405 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bbcc1-5bfa-403c-a41b-590d077aa800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:48.003776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bbcc1-5bfa-403c-a41b-590d077aa800 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:48.004580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23bbcc1-5bfa-403c-a41b-590d077aa800 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:48.006855 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: WARNING py.warnings [None req-e23bbcc1-5bfa-403c-a41b-590d077aa800 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:27:48.006855 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: warnings.warn(msg) Jul 20 15:27:48.006855 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: Jul 20 15:27:48.008871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23bbcc1-5bfa-403c-a41b-590d077aa800 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:48.134981 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 742/1497] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:27:47 2022] POST /identity/v3/projects => generated 377 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:27:48.139775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c3806-90b4-45ed-b3b8-7c0c01a9aa92 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:48.141355 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-617c3806-90b4-45ed-b3b8-7c0c01a9aa92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:48.178230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c3806-90b4-45ed-b3b8-7c0c01a9aa92 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:48.287123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac869ad9-0d8c-495c-b139-4f0db08c2af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:48.288739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac869ad9-0d8c-495c-b139-4f0db08c2af5 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:48.289749 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac869ad9-0d8c-495c-b139-4f0db08c2af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:48.290644 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac869ad9-0d8c-495c-b139-4f0db08c2af5 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:48.295623 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac869ad9-0d8c-495c-b139-4f0db08c2af5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:48.297256 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: WARNING py.warnings [None req-ac869ad9-0d8c-495c-b139-4f0db08c2af5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:27:48.297256 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: warnings.warn(msg) Jul 20 15:27:48.297256 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: Jul 20 15:27:48.298678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac869ad9-0d8c-495c-b139-4f0db08c2af5 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:48.419352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 756/1498] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:27:48 2022] POST /identity/v3/users => generated 289 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:27:48.424649 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23bbcc1-5bfa-403c-a41b-590d077aa800 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:48.426306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e23bbcc1-5bfa-403c-a41b-590d077aa800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:48.501185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23bbcc1-5bfa-403c-a41b-590d077aa800 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:48.594478 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:48.606528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b18db8-ea2f-44a5-ad2a-66b433e629f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:48.607067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b18db8-ea2f-44a5-ad2a-66b433e629f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:48.607348 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b18db8-ea2f-44a5-ad2a-66b433e629f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:48.607618 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b18db8-ea2f-44a5-ad2a-66b433e629f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:48.608352 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b18db8-ea2f-44a5-ad2a-66b433e629f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:48.609410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: WARNING py.warnings [None req-43b18db8-ea2f-44a5-ad2a-66b433e629f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:27:48.609410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: warnings.warn(msg) Jul 20 15:27:48.609410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: Jul 20 15:27:48.610304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b18db8-ea2f-44a5-ad2a-66b433e629f6 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:48.623826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 743/1499] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:27:48.628343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac869ad9-0d8c-495c-b139-4f0db08c2af5 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:48.630080 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ac869ad9-0d8c-495c-b139-4f0db08c2af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:48.642705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac869ad9-0d8c-495c-b139-4f0db08c2af5 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:48.674095 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:48.754221 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba06721-638a-4a98-94b7-970fe2d5ccc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:48.755091 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba06721-638a-4a98-94b7-970fe2d5ccc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:48.755458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba06721-638a-4a98-94b7-970fe2d5ccc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:48.755829 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba06721-638a-4a98-94b7-970fe2d5ccc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:48.756753 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba06721-638a-4a98-94b7-970fe2d5ccc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:48.757869 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: WARNING py.warnings [None req-9ba06721-638a-4a98-94b7-970fe2d5ccc9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:27:48.757869 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: warnings.warn(msg) Jul 20 15:27:48.757869 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: Jul 20 15:27:48.759021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba06721-638a-4a98-94b7-970fe2d5ccc9 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:48.776074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 757/1500] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:27:48.782047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b18db8-ea2f-44a5-ad2a-66b433e629f6 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:48.783618 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-43b18db8-ea2f-44a5-ad2a-66b433e629f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:48.819759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b18db8-ea2f-44a5-ad2a-66b433e629f6 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:48.920451 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7b166-e287-489a-bb88-286f63a0ba06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:48.921286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7b166-e287-489a-bb88-286f63a0ba06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:48.921575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7b166-e287-489a-bb88-286f63a0ba06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:48.921852 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7b166-e287-489a-bb88-286f63a0ba06 admin admin] PATH_INFO: `/v3/projects/0a44ba4d82824c24992f226c305864ec/users/30bc8cffe56341dfbe05f62c83759d84/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:48.958060 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce7b166-e287-489a-bb88-286f63a0ba06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a44ba4d82824c24992f226c305864ec, user_id=30bc8cffe56341dfbe05f62c83759d84, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:48.959148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: WARNING py.warnings [None req-7ce7b166-e287-489a-bb88-286f63a0ba06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:27:48.959148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: warnings.warn(msg) Jul 20 15:27:48.959148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: Jul 20 15:27:48.961140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce7b166-e287-489a-bb88-286f63a0ba06 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:49.069185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 744/1501] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:27:48 2022] PUT /identity/v3/projects/0a44ba4d82824c24992f226c305864ec/users/30bc8cffe56341dfbe05f62c83759d84/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:27:49.082458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bccd53-03fe-4764-9424-d1ac33d00587 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:49.082744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bccd53-03fe-4764-9424-d1ac33d00587 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:49.083024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bccd53-03fe-4764-9424-d1ac33d00587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:49.154891 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-44bccd53-03fe-4764-9424-d1ac33d00587 None None] MFA Rules not processed for user `30bc8cffe56341dfbe05f62c83759d84`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:27:49.156184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-44bccd53-03fe-4764-9424-d1ac33d00587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:49.259736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 758/1502] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:27:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:27:49.274010 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06cd23-8218-4ed7-9656-0ca0ad424445 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:49.274413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06cd23-8218-4ed7-9656-0ca0ad424445 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:49.274701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06cd23-8218-4ed7-9656-0ca0ad424445 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:49.276095 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 745/1503] 10.222.0.78 () {54 vars in 964 bytes} [Wed Jul 20 15:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:27:49.281458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e730301-1a92-4925-b37a-d482defd63cd None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:49.281801 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e730301-1a92-4925-b37a-d482defd63cd None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:49.282043 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e730301-1a92-4925-b37a-d482defd63cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:49.313394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:49.362617 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6e730301-1a92-4925-b37a-d482defd63cd None None] MFA Rules not processed for user `b51e9bdab4164d87b7b0b11c505c2c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:27:49.363846 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6e730301-1a92-4925-b37a-d482defd63cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:49.463981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 759/1504] 10.222.0.78 () {58 vars in 1061 bytes} [Wed Jul 20 15:27:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:27:49.472051 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06cd23-8218-4ed7-9656-0ca0ad424445 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:49.473623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-bf06cd23-8218-4ed7-9656-0ca0ad424445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:49.529159 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:49.599250 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06cd23-8218-4ed7-9656-0ca0ad424445 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:49.692372 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154443d5-8ce7-440e-b904-c202d13536e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:49.693401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154443d5-8ce7-440e-b904-c202d13536e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:49.693694 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154443d5-8ce7-440e-b904-c202d13536e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:49.693972 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154443d5-8ce7-440e-b904-c202d13536e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:49.695689 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-154443d5-8ce7-440e-b904-c202d13536e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:49.825525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154443d5-8ce7-440e-b904-c202d13536e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:49.828217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154443d5-8ce7-440e-b904-c202d13536e9 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:49.834332 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:49.850146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 746/1505] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:27:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:27:49.895480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585230c0-c742-46c0-ada7-5f2a56c11bf7 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:49.896197 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585230c0-c742-46c0-ada7-5f2a56c11bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:49.900663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585230c0-c742-46c0-ada7-5f2a56c11bf7 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:49.902162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 760/1506] 10.222.0.78 () {52 vars in 829 bytes} [Wed Jul 20 15:27:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:27:49.906886 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154443d5-8ce7-440e-b904-c202d13536e9 service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:49.908128 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-154443d5-8ce7-440e-b904-c202d13536e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:50.028098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154443d5-8ce7-440e-b904-c202d13536e9 service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:50.134759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e4e69-f65e-40e2-bd2e-9b77013cf07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:50.135757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e4e69-f65e-40e2-bd2e-9b77013cf07a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:50.136152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e4e69-f65e-40e2-bd2e-9b77013cf07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:50.136523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e4e69-f65e-40e2-bd2e-9b77013cf07a admin admin] PATH_INFO: `/v3/users/30bc8cffe56341dfbe05f62c83759d84` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:50.150979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e4e69-f65e-40e2-bd2e-9b77013cf07a admin admin] RBAC: Authorizing `identity:delete_user(user_id=30bc8cffe56341dfbe05f62c83759d84)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:50.152310 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: WARNING py.warnings [None req-4f3e4e69-f65e-40e2-bd2e-9b77013cf07a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:27:50.152310 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: warnings.warn(msg) Jul 20 15:27:50.152310 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: Jul 20 15:27:50.153668 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e4e69-f65e-40e2-bd2e-9b77013cf07a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:50.168421 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:50.386728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-4f3e4e69-f65e-40e2-bd2e-9b77013cf07a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30bc8cffe56341dfbe05f62c83759d84'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:27:50.452808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-4f3e4e69-f65e-40e2-bd2e-9b77013cf07a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30bc8cffe56341dfbe05f62c83759d84'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:27:50.453286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-4f3e4e69-f65e-40e2-bd2e-9b77013cf07a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30bc8cffe56341dfbe05f62c83759d84'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:27:50.479922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-4f3e4e69-f65e-40e2-bd2e-9b77013cf07a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30bc8cffe56341dfbe05f62c83759d84'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:27:50.496355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-4f3e4e69-f65e-40e2-bd2e-9b77013cf07a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30bc8cffe56341dfbe05f62c83759d84'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:27:50.502331 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 747/1507] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:27:49 2022] DELETE /identity/v3/users/30bc8cffe56341dfbe05f62c83759d84 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:27:50.507088 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb291df6-f9de-4e8e-8dd1-ecbe44c26ca6 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:50.507832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb291df6-f9de-4e8e-8dd1-ecbe44c26ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:50.507832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb291df6-f9de-4e8e-8dd1-ecbe44c26ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:50.574225 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-bb291df6-f9de-4e8e-8dd1-ecbe44c26ca6 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:27:50.576375 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-bb291df6-f9de-4e8e-8dd1-ecbe44c26ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:50.654147 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:50.667922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 761/1508] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:27:50.678332 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e4e69-f65e-40e2-bd2e-9b77013cf07a admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:50.679945 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4f3e4e69-f65e-40e2-bd2e-9b77013cf07a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:50.800972 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e4e69-f65e-40e2-bd2e-9b77013cf07a admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:50.862064 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:50.882714 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e8e67-01ec-4489-9611-e12f4f88534f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:50.883433 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e8e67-01ec-4489-9611-e12f4f88534f service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:50.883720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e8e67-01ec-4489-9611-e12f4f88534f service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:50.884033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e8e67-01ec-4489-9611-e12f4f88534f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:50.885739 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-439e8e67-01ec-4489-9611-e12f4f88534f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:51.010220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e8e67-01ec-4489-9611-e12f4f88534f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:51.011728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e8e67-01ec-4489-9611-e12f4f88534f service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:51.032289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 748/1509] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:27:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:27:51.589806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb291df6-f9de-4e8e-8dd1-ecbe44c26ca6 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:27:51.598719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-bb291df6-f9de-4e8e-8dd1-ecbe44c26ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:27:51.610233 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb291df6-f9de-4e8e-8dd1-ecbe44c26ca6 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:27:51.731160 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d7fd9-f107-408e-9c7e-1757fddd1065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:27:51.732035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d7fd9-f107-408e-9c7e-1757fddd1065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:27:51.732317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d7fd9-f107-408e-9c7e-1757fddd1065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:27:51.732583 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d7fd9-f107-408e-9c7e-1757fddd1065 admin admin] PATH_INFO: `/v3/projects/0a44ba4d82824c24992f226c305864ec` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:27:51.745317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d7fd9-f107-408e-9c7e-1757fddd1065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a44ba4d82824c24992f226c305864ec)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:27:51.746384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: WARNING py.warnings [None req-f37d7fd9-f107-408e-9c7e-1757fddd1065 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:27:51.746384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: warnings.warn(msg) Jul 20 15:27:51.746384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: Jul 20 15:27:51.747572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d7fd9-f107-408e-9c7e-1757fddd1065 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:27:51.900237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f37d7fd9-f107-408e-9c7e-1757fddd1065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a44ba4d82824c24992f226c305864ec', 'request_id': 'req-f37d7fd9-f107-408e-9c7e-1757fddd1065'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:27:51.975046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f37d7fd9-f107-408e-9c7e-1757fddd1065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a44ba4d82824c24992f226c305864ec', 'request_id': 'req-f37d7fd9-f107-408e-9c7e-1757fddd1065'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:27:51.990395 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f37d7fd9-f107-408e-9c7e-1757fddd1065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a44ba4d82824c24992f226c305864ec', 'request_id': 'req-f37d7fd9-f107-408e-9c7e-1757fddd1065'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:27:52.017263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f37d7fd9-f107-408e-9c7e-1757fddd1065 admin admin] The token cache is being invalidate because project 0a44ba4d82824c24992f226c305864ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:27:52.017555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f37d7fd9-f107-408e-9c7e-1757fddd1065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:27:52.018082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f37d7fd9-f107-408e-9c7e-1757fddd1065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:27:52.019598 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 762/1510] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:27:51 2022] DELETE /identity/v3/projects/0a44ba4d82824c24992f226c305864ec => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:28:01.029548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=62 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:02.015089 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:32.620097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a3e00-8a3c-4a64-92df-07e82675ebea None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:32.621014 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a3e00-8a3c-4a64-92df-07e82675ebea None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:32.621014 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a3e00-8a3c-4a64-92df-07e82675ebea None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:32.622644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:29:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:29:32.629012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9f3b8-0088-4da2-8216-22c8aad6955f None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:32.639636 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9f3b8-0088-4da2-8216-22c8aad6955f None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:32.639636 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9f3b8-0088-4da2-8216-22c8aad6955f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:32.732614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-60a9f3b8-0088-4da2-8216-22c8aad6955f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:32.733876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-60a9f3b8-0088-4da2-8216-22c8aad6955f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:32.851179 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:33.008310 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a3e00-8a3c-4a64-92df-07e82675ebea None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:33.011281 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-dd6a3e00-8a3c-4a64-92df-07e82675ebea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:33.162940 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a3e00-8a3c-4a64-92df-07e82675ebea None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:33.168605 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93af52-2fb5-4e7f-a0e4-d66570cacac3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:33.169261 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93af52-2fb5-4e7f-a0e4-d66570cacac3 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:33.169544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93af52-2fb5-4e7f-a0e4-d66570cacac3 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:33.169829 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93af52-2fb5-4e7f-a0e4-d66570cacac3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:33.180185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93af52-2fb5-4e7f-a0e4-d66570cacac3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:33.182602 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93af52-2fb5-4e7f-a0e4-d66570cacac3 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:33.197456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 750/1513] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:34.109731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63a0b6-5a1f-45f6-a3ac-8cad12ad80e0 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:34.110364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63a0b6-5a1f-45f6-a3ac-8cad12ad80e0 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:34.110670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63a0b6-5a1f-45f6-a3ac-8cad12ad80e0 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:34.112990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:29:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:34.120032 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401a5ed-8ff6-49e7-909c-cca923e0357d None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:34.120593 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401a5ed-8ff6-49e7-909c-cca923e0357d None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:34.121007 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401a5ed-8ff6-49e7-909c-cca923e0357d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:34.149790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-7401a5ed-8ff6-49e7-909c-cca923e0357d None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:34.151722 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-7401a5ed-8ff6-49e7-909c-cca923e0357d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:34.164323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:34.355513 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63a0b6-5a1f-45f6-a3ac-8cad12ad80e0 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:34.373703 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63a0b6-5a1f-45f6-a3ac-8cad12ad80e0 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:34.381300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f4ba6-17a4-4eae-b1d1-8b1d95631a7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:34.381966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f4ba6-17a4-4eae-b1d1-8b1d95631a7e service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:34.382290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f4ba6-17a4-4eae-b1d1-8b1d95631a7e service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:34.382568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f4ba6-17a4-4eae-b1d1-8b1d95631a7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:34.392405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f4ba6-17a4-4eae-b1d1-8b1d95631a7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:34.394641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f4ba6-17a4-4eae-b1d1-8b1d95631a7e service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:34.405824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 765/1516] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:34.686537 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609cd1a-657b-4fc7-83de-26a3509b7180 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:34.686775 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609cd1a-657b-4fc7-83de-26a3509b7180 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:34.687114 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609cd1a-657b-4fc7-83de-26a3509b7180 None None] PATH_INFO: `/v3` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:34.688558 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 752/1517] 10.222.0.102 () {54 vars in 926 bytes} [Wed Jul 20 15:29:34 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:29:34.694854 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee23d99-c771-449e-af80-35f230550221 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:34.695193 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee23d99-c771-449e-af80-35f230550221 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:34.695521 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee23d99-c771-449e-af80-35f230550221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:34.766959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-2ee23d99-c771-449e-af80-35f230550221 None None] MFA Rules not processed for user `2192547babde4449a46f9cb37ed64524`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:34.768766 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2ee23d99-c771-449e-af80-35f230550221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:34.794083 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:34.865970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 766/1518] 10.222.0.102 () {58 vars in 1016 bytes} [Wed Jul 20 15:29:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:39.229267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae7308-7b75-4193-8d3b-1c0b24ad7706 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:39.229837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae7308-7b75-4193-8d3b-1c0b24ad7706 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:39.230017 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae7308-7b75-4193-8d3b-1c0b24ad7706 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:39.232194 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 753/1519] 10.222.0.78 () {54 vars in 961 bytes} [Wed Jul 20 15:29:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:39.238234 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6564e-32c5-4e72-9a2a-75a6a1646193 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:39.238891 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6564e-32c5-4e72-9a2a-75a6a1646193 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:39.239103 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6564e-32c5-4e72-9a2a-75a6a1646193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:39.265852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-68b6564e-32c5-4e72-9a2a-75a6a1646193 None None] MFA Rules not processed for user `2192547babde4449a46f9cb37ed64524`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:39.267247 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-68b6564e-32c5-4e72-9a2a-75a6a1646193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:39.288195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 767/1520] 10.222.0.78 () {58 vars in 1058 bytes} [Wed Jul 20 15:29:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:39.297962 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae7308-7b75-4193-8d3b-1c0b24ad7706 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:39.313044 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae7308-7b75-4193-8d3b-1c0b24ad7706 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:39.318685 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f98e7-ccd1-4a6c-9800-431852da7f1e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2192547babde4449a46f9cb37ed64524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:39.319502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f98e7-ccd1-4a6c-9800-431852da7f1e service placement] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:39.319795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f98e7-ccd1-4a6c-9800-431852da7f1e service placement] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:39.320115 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f98e7-ccd1-4a6c-9800-431852da7f1e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:39.330121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f98e7-ccd1-4a6c-9800-431852da7f1e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:39.331775 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f98e7-ccd1-4a6c-9800-431852da7f1e service placement] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:39.343968 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 754/1521] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:45.756462 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360fa65-802c-4f7e-9ef7-ef9c91a447b8 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:45.757351 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360fa65-802c-4f7e-9ef7-ef9c91a447b8 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:45.757351 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360fa65-802c-4f7e-9ef7-ef9c91a447b8 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:45.758420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:29:45.765335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b30ae-7cdb-4f6b-b77a-6a1936133d2c None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:45.765907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b30ae-7cdb-4f6b-b77a-6a1936133d2c None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:45.766169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b30ae-7cdb-4f6b-b77a-6a1936133d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:45.797236 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-b78b30ae-7cdb-4f6b-b77a-6a1936133d2c None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:45.799450 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b78b30ae-7cdb-4f6b-b77a-6a1936133d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:45.813582 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:29:45.975878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360fa65-802c-4f7e-9ef7-ef9c91a447b8 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:45.992311 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360fa65-802c-4f7e-9ef7-ef9c91a447b8 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:45.998872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd593c9-5e32-402e-b971-2eba358e832d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:45.999542 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd593c9-5e32-402e-b971-2eba358e832d service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:45.999835 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd593c9-5e32-402e-b971-2eba358e832d service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:46.000110 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd593c9-5e32-402e-b971-2eba358e832d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:46.011304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd593c9-5e32-402e-b971-2eba358e832d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:46.014012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd593c9-5e32-402e-b971-2eba358e832d service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:46.025446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 769/1524] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:46.989904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3cd00-adc2-449c-9449-a733d7e213d7 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:46.990792 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3cd00-adc2-449c-9449-a733d7e213d7 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:46.990792 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3cd00-adc2-449c-9449-a733d7e213d7 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:46.991899 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 15:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:46.999192 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510eb7f4-2f5c-4834-bfd3-edb0116376f4 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:47.000012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510eb7f4-2f5c-4834-bfd3-edb0116376f4 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:47.000012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510eb7f4-2f5c-4834-bfd3-edb0116376f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:47.023987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-510eb7f4-2f5c-4834-bfd3-edb0116376f4 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:47.025188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-510eb7f4-2f5c-4834-bfd3-edb0116376f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:47.034423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 15:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:47.239132 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3cd00-adc2-449c-9449-a733d7e213d7 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:47.264180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3cd00-adc2-449c-9449-a733d7e213d7 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:47.275348 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a077be-f03c-4f9d-ae19-a0d8094bd43f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:47.276018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a077be-f03c-4f9d-ae19-a0d8094bd43f service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:47.276300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a077be-f03c-4f9d-ae19-a0d8094bd43f service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:47.276582 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a077be-f03c-4f9d-ae19-a0d8094bd43f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:47.286393 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a077be-f03c-4f9d-ae19-a0d8094bd43f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:47.288048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a077be-f03c-4f9d-ae19-a0d8094bd43f service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:47.300190 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 757/1527] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:29:51.392058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47bd9f-4478-45b3-bd86-67ea0b1317aa None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:51.392575 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47bd9f-4478-45b3-bd86-67ea0b1317aa None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:51.392575 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47bd9f-4478-45b3-bd86-67ea0b1317aa None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:51.393943 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:29:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:51.399793 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de9ecc-51d7-4515-b228-7f0f307faa94 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:51.400613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de9ecc-51d7-4515-b228-7f0f307faa94 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:51.400613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de9ecc-51d7-4515-b228-7f0f307faa94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:51.419602 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-26de9ecc-51d7-4515-b228-7f0f307faa94 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:51.420823 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-26de9ecc-51d7-4515-b228-7f0f307faa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:51.428940 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:29:51.522475 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47bd9f-4478-45b3-bd86-67ea0b1317aa None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:51.524238 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-3f47bd9f-4478-45b3-bd86-67ea0b1317aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:51.624786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47bd9f-4478-45b3-bd86-67ea0b1317aa None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:51.628759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5e6c9-4849-4eae-bcb9-4a61cf86d503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:51.629420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5e6c9-4849-4eae-bcb9-4a61cf86d503 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:51.629701 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5e6c9-4849-4eae-bcb9-4a61cf86d503 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:51.629975 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5e6c9-4849-4eae-bcb9-4a61cf86d503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:51.640787 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5e6c9-4849-4eae-bcb9-4a61cf86d503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:51.642345 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5e6c9-4849-4eae-bcb9-4a61cf86d503 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:51.653360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 772/1530] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:29:53.990474 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ce9ad-c19a-47e9-9979-16b73a974f72 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:53.991030 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ce9ad-c19a-47e9-9979-16b73a974f72 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:53.991271 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ce9ad-c19a-47e9-9979-16b73a974f72 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:53.992780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:29:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:53.998802 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea9161-fe5e-4288-8acf-4e82cebcb4bd None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:53.999509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea9161-fe5e-4288-8acf-4e82cebcb4bd None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:53.999509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea9161-fe5e-4288-8acf-4e82cebcb4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:54.023124 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6eea9161-fe5e-4288-8acf-4e82cebcb4bd None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:54.024387 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6eea9161-fe5e-4288-8acf-4e82cebcb4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:54.033925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:29:54.077551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ce9ad-c19a-47e9-9979-16b73a974f72 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:54.090848 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ce9ad-c19a-47e9-9979-16b73a974f72 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:54.095993 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01e330-0b69-43e3-9ce0-66b4ab32510c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:54.097294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01e330-0b69-43e3-9ce0-66b4ab32510c service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:54.097637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01e330-0b69-43e3-9ce0-66b4ab32510c service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:54.098099 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01e330-0b69-43e3-9ce0-66b4ab32510c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:54.112031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01e330-0b69-43e3-9ce0-66b4ab32510c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:54.113548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01e330-0b69-43e3-9ce0-66b4ab32510c service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:54.124646 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 760/1533] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:29:57.940117 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bcfa5-8238-4345-965b-640828a3c58d None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:57.940880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bcfa5-8238-4345-965b-640828a3c58d None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:57.940880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bcfa5-8238-4345-965b-640828a3c58d None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:57.942101 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:29:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:29:57.948297 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5dfc9-b31f-4ecf-bd70-2ab8d7702323 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:57.948818 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5dfc9-b31f-4ecf-bd70-2ab8d7702323 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:57.948930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5dfc9-b31f-4ecf-bd70-2ab8d7702323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:57.969023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-eae5dfc9-b31f-4ecf-bd70-2ab8d7702323 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:29:57.970415 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-eae5dfc9-b31f-4ecf-bd70-2ab8d7702323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:57.981154 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:29:58.301994 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1bcfa5-8238-4345-965b-640828a3c58d None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:29:58.305777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6c1bcfa5-8238-4345-965b-640828a3c58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:29:58.401099 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:58.415628 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1bcfa5-8238-4345-965b-640828a3c58d None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:29:58.420566 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5521e2-dcc4-4892-83d4-8a33ca74fff9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:29:58.421235 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5521e2-dcc4-4892-83d4-8a33ca74fff9 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:29:58.421516 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5521e2-dcc4-4892-83d4-8a33ca74fff9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:29:58.421796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5521e2-dcc4-4892-83d4-8a33ca74fff9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:29:58.431704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5521e2-dcc4-4892-83d4-8a33ca74fff9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:29:58.433151 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5521e2-dcc4-4892-83d4-8a33ca74fff9 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:29:58.444734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 775/1536] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:29:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:30:00.570319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5d179-3e9b-4fae-a803-dd0ba5e3dff7 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:00.570876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5d179-3e9b-4fae-a803-dd0ba5e3dff7 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:00.570876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5d179-3e9b-4fae-a803-dd0ba5e3dff7 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:00.572382 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 15:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:30:00.578180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9eba5-b92d-4185-ac0a-075051a17650 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:00.578705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9eba5-b92d-4185-ac0a-075051a17650 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:00.578837 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9eba5-b92d-4185-ac0a-075051a17650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:00.597729 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-1ad9eba5-b92d-4185-ac0a-075051a17650 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:30:00.598982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1ad9eba5-b92d-4185-ac0a-075051a17650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:00.607664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 15:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:30:00.926790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa5d179-3e9b-4fae-a803-dd0ba5e3dff7 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:30:00.928721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-faa5d179-3e9b-4fae-a803-dd0ba5e3dff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:30:00.942641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa5d179-3e9b-4fae-a803-dd0ba5e3dff7 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:30:00.946468 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fca37-2ec6-4178-bd04-18b7319d7f73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:30:00.947137 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fca37-2ec6-4178-bd04-18b7319d7f73 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:30:00.947415 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fca37-2ec6-4178-bd04-18b7319d7f73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:30:00.947691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fca37-2ec6-4178-bd04-18b7319d7f73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:30:00.959371 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1fca37-2ec6-4178-bd04-18b7319d7f73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:30:00.960818 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1fca37-2ec6-4178-bd04-18b7319d7f73 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:30:00.973285 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 763/1539] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:30:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:30:10.592069 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=11 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:10.967862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:32.089685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb746f51-6574-41e9-9cf9-576de33b52a7 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:32.090413 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb746f51-6574-41e9-9cf9-576de33b52a7 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:32.090749 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb746f51-6574-41e9-9cf9-576de33b52a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:32.134832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-cb746f51-6574-41e9-9cf9-576de33b52a7 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:32.139578 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-cb746f51-6574-41e9-9cf9-576de33b52a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:32.158147 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 777/1540] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:32.171639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8d569-e3a3-42bc-8a05-c21785594d23 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:32.172609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8d569-e3a3-42bc-8a05-c21785594d23 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:32.173104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8d569-e3a3-42bc-8a05-c21785594d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:32.206801 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-75a8d569-e3a3-42bc-8a05-c21785594d23 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:32.208042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-75a8d569-e3a3-42bc-8a05-c21785594d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:32.218305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 764/1541] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:31:32.225532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb746f51-6574-41e9-9cf9-576de33b52a7 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:32.254353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb746f51-6574-41e9-9cf9-576de33b52a7 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:32.266043 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbe28b-1fb2-4f54-b6f6-38c933cbf970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:32.267249 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbe28b-1fb2-4f54-b6f6-38c933cbf970 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:32.267739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbe28b-1fb2-4f54-b6f6-38c933cbf970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:32.268232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbe28b-1fb2-4f54-b6f6-38c933cbf970 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:32.270629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbe28b-1fb2-4f54-b6f6-38c933cbf970 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:31:32.271201 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbe28b-1fb2-4f54-b6f6-38c933cbf970 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:32.281963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbe28b-1fb2-4f54-b6f6-38c933cbf970 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:32.316528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 778/1542] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:31:32.328072 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8d569-e3a3-42bc-8a05-c21785594d23 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:32.346205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8d569-e3a3-42bc-8a05-c21785594d23 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:32.355652 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f4d6f-3943-44db-882a-61b07fd3f82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:32.356825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f4d6f-3943-44db-882a-61b07fd3f82a admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:32.357336 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f4d6f-3943-44db-882a-61b07fd3f82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:32.357835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f4d6f-3943-44db-882a-61b07fd3f82a admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:32.359292 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f4d6f-3943-44db-882a-61b07fd3f82a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:32.362946 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f4d6f-3943-44db-882a-61b07fd3f82a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:32.522509 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 765/1543] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:31:32 2022] POST /identity/v3/projects => generated 371 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:31:32.533521 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbe28b-1fb2-4f54-b6f6-38c933cbf970 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:32.563329 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbe28b-1fb2-4f54-b6f6-38c933cbf970 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:32.708191 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad6862-dc56-44c0-82bc-fda4db7cc9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:32.709266 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad6862-dc56-44c0-82bc-fda4db7cc9ae admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:32.709764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad6862-dc56-44c0-82bc-fda4db7cc9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:32.710276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad6862-dc56-44c0-82bc-fda4db7cc9ae admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:32.711511 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad6862-dc56-44c0-82bc-fda4db7cc9ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:32.715782 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad6862-dc56-44c0-82bc-fda4db7cc9ae admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:32.781713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813b5c1-f099-4380-8036-7b931e21cb7f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:32.782021 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813b5c1-f099-4380-8036-7b931e21cb7f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:32.782384 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813b5c1-f099-4380-8036-7b931e21cb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:32.819634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-8813b5c1-f099-4380-8036-7b931e21cb7f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:32.820906 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8813b5c1-f099-4380-8036-7b931e21cb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:32.836379 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 766/1544] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:32.847788 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837697b4-daf3-4da6-abd8-56ee5513286c None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:32.848055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837697b4-daf3-4da6-abd8-56ee5513286c None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:32.848349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837697b4-daf3-4da6-abd8-56ee5513286c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:32.873423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 779/1545] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:31:32 2022] POST /identity/v3/users => generated 286 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:32.877883 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-837697b4-daf3-4da6-abd8-56ee5513286c None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:32.877883 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-837697b4-daf3-4da6-abd8-56ee5513286c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:32.883258 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad6862-dc56-44c0-82bc-fda4db7cc9ae admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:32.888167 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 767/1546] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:31:32.900228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80617e-e06a-4332-b62c-8079a49723f5 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:32.900529 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80617e-e06a-4332-b62c-8079a49723f5 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:32.900814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80617e-e06a-4332-b62c-8079a49723f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:32.911111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad6862-dc56-44c0-82bc-fda4db7cc9ae admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:32.925585 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe85acc-7ed5-4b1d-869d-5ed2396fc458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:32.927392 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe85acc-7ed5-4b1d-869d-5ed2396fc458 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:32.927392 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe85acc-7ed5-4b1d-869d-5ed2396fc458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:32.927777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe85acc-7ed5-4b1d-869d-5ed2396fc458 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:32.929132 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe85acc-7ed5-4b1d-869d-5ed2396fc458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:32.933126 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe85acc-7ed5-4b1d-869d-5ed2396fc458 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:32.935388 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-bd80617e-e06a-4332-b62c-8079a49723f5 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:32.936644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-bd80617e-e06a-4332-b62c-8079a49723f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:32.967302 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 768/1547] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:32.968496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80617e-e06a-4332-b62c-8079a49723f5 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:32.971075 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 780/1548] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:32.976743 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe85acc-7ed5-4b1d-869d-5ed2396fc458 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:32.981758 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80617e-e06a-4332-b62c-8079a49723f5 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:33.008320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe85acc-7ed5-4b1d-869d-5ed2396fc458 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:33.011153 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038eecef-482a-412c-be76-97bbda547f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:33.011980 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eecef-482a-412c-be76-97bbda547f8b admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.012199 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eecef-482a-412c-be76-97bbda547f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.012469 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eecef-482a-412c-be76-97bbda547f8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.013229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038eecef-482a-412c-be76-97bbda547f8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:31:33.013544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038eecef-482a-412c-be76-97bbda547f8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:33.014614 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: WARNING py.warnings [None req-038eecef-482a-412c-be76-97bbda547f8b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:31:33.014614 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: warnings.warn(msg) Jul 20 15:31:33.014614 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: Jul 20 15:31:33.015543 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038eecef-482a-412c-be76-97bbda547f8b admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:33.021951 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334d37e-9d9e-4d8e-9c8b-ecdd6991abc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:33.023147 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334d37e-9d9e-4d8e-9c8b-ecdd6991abc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.023647 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334d37e-9d9e-4d8e-9c8b-ecdd6991abc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.024114 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334d37e-9d9e-4d8e-9c8b-ecdd6991abc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.026833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334d37e-9d9e-4d8e-9c8b-ecdd6991abc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:33.029820 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334d37e-9d9e-4d8e-9c8b-ecdd6991abc4 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:33.034028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 769/1549] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:31:33.037075 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc64f34-37cc-49e2-a4ed-efc04dee97ab None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.037365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc64f34-37cc-49e2-a4ed-efc04dee97ab None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.037653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc64f34-37cc-49e2-a4ed-efc04dee97ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.073868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 781/1550] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:33.075109 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-acc64f34-37cc-49e2-a4ed-efc04dee97ab None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:33.075278 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334d37e-9d9e-4d8e-9c8b-ecdd6991abc4 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:33.076477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-acc64f34-37cc-49e2-a4ed-efc04dee97ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:33.109774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334d37e-9d9e-4d8e-9c8b-ecdd6991abc4 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:33.116338 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 770/1551] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:33.117334 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc64f34-37cc-49e2-a4ed-efc04dee97ab None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:33.130077 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17aab4-32c3-4f26-99e0-a45ffaa405eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:33.130816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17aab4-32c3-4f26-99e0-a45ffaa405eb admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.131105 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17aab4-32c3-4f26-99e0-a45ffaa405eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.131393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17aab4-32c3-4f26-99e0-a45ffaa405eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.132150 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17aab4-32c3-4f26-99e0-a45ffaa405eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:33.133223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: WARNING py.warnings [None req-0e17aab4-32c3-4f26-99e0-a45ffaa405eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:31:33.133223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: warnings.warn(msg) Jul 20 15:31:33.133223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: Jul 20 15:31:33.134480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17aab4-32c3-4f26-99e0-a45ffaa405eb admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:33.146726 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc64f34-37cc-49e2-a4ed-efc04dee97ab None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:33.168373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e198c-9572-4b05-bc51-71f2c8fb79b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:33.169319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e198c-9572-4b05-bc51-71f2c8fb79b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.169607 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e198c-9572-4b05-bc51-71f2c8fb79b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.169888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e198c-9572-4b05-bc51-71f2c8fb79b4 admin admin] PATH_INFO: `/v3/projects/f10025ba797e4e21a8ebf27435eea153/users/f0b4d325f04847158bffc4c8d8fdfe48/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.201899 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e198c-9572-4b05-bc51-71f2c8fb79b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f10025ba797e4e21a8ebf27435eea153, user_id=f0b4d325f04847158bffc4c8d8fdfe48, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:33.206842 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e198c-9572-4b05-bc51-71f2c8fb79b4 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:33.311164 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 782/1552] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:31:33 2022] POST /identity/v3/projects => generated 377 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:33.312837 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17aab4-32c3-4f26-99e0-a45ffaa405eb admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:33.361921 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 771/1553] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:33 2022] PUT /identity/v3/projects/f10025ba797e4e21a8ebf27435eea153/users/f0b4d325f04847158bffc4c8d8fdfe48/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:33.368237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e198c-9572-4b05-bc51-71f2c8fb79b4 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:33.478044 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e198c-9572-4b05-bc51-71f2c8fb79b4 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:33.492145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7b694-e964-43a6-a88a-5b77fb36da79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:33.493764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7b694-e964-43a6-a88a-5b77fb36da79 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.495599 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7b694-e964-43a6-a88a-5b77fb36da79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.496106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7b694-e964-43a6-a88a-5b77fb36da79 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.498345 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7b694-e964-43a6-a88a-5b77fb36da79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:33.499395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: WARNING py.warnings [None req-35c7b694-e964-43a6-a88a-5b77fb36da79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:31:33.499395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: warnings.warn(msg) Jul 20 15:31:33.499395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: Jul 20 15:31:33.501580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7b694-e964-43a6-a88a-5b77fb36da79 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:33.515460 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17aab4-32c3-4f26-99e0-a45ffaa405eb admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:33.527522 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a304e-05dd-472d-a610-1f1b9338c5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:33.528527 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a304e-05dd-472d-a610-1f1b9338c5ea admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.528912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a304e-05dd-472d-a610-1f1b9338c5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.529285 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a304e-05dd-472d-a610-1f1b9338c5ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.530156 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a304e-05dd-472d-a610-1f1b9338c5ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:31:33.530647 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a304e-05dd-472d-a610-1f1b9338c5ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:33.534319 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a304e-05dd-472d-a610-1f1b9338c5ea admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:33.548774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 783/1554] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:31:33.550447 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e3b04-22d5-4936-9500-b5533bfa2489 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.550842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e3b04-22d5-4936-9500-b5533bfa2489 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.551224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e3b04-22d5-4936-9500-b5533bfa2489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.575114 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-0d1e3b04-22d5-4936-9500-b5533bfa2489 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:33.576760 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0d1e3b04-22d5-4936-9500-b5533bfa2489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:33.586133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 784/1555] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:33.589445 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152482e-f124-43e1-8ec6-a8321e3cd021 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.589445 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152482e-f124-43e1-8ec6-a8321e3cd021 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.589445 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152482e-f124-43e1-8ec6-a8321e3cd021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.616608 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:33.620440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6152482e-f124-43e1-8ec6-a8321e3cd021 None None] MFA Rules not processed for user `f0b4d325f04847158bffc4c8d8fdfe48`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:33.626067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6152482e-f124-43e1-8ec6-a8321e3cd021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:33.662849 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 772/1556] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:31:33 2022] POST /identity/v3/users => generated 289 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:33.666246 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d12e3-2218-45f8-a341-32ba6555d5bd None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.666246 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d12e3-2218-45f8-a341-32ba6555d5bd None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.666246 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d12e3-2218-45f8-a341-32ba6555d5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.688559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-9b7d12e3-2218-45f8-a341-32ba6555d5bd None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:33.690284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-9b7d12e3-2218-45f8-a341-32ba6555d5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:33.694123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 785/1557] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:33.698804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152482e-f124-43e1-8ec6-a8321e3cd021 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:33.703358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 773/1558] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:33.704717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca9624-873d-4bf3-9a3b-3779a55451c4 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.705002 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca9624-873d-4bf3-9a3b-3779a55451c4 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.705284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca9624-873d-4bf3-9a3b-3779a55451c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.709819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152482e-f124-43e1-8ec6-a8321e3cd021 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:33.717653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0205d-e7ac-477b-8584-f4292cc17662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:33.719346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0205d-e7ac-477b-8584-f4292cc17662 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.723481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0205d-e7ac-477b-8584-f4292cc17662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.723481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0205d-e7ac-477b-8584-f4292cc17662 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.723481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0205d-e7ac-477b-8584-f4292cc17662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:33.723481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0205d-e7ac-477b-8584-f4292cc17662 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:33.731990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-d2ca9624-873d-4bf3-9a3b-3779a55451c4 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:33.733228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d2ca9624-873d-4bf3-9a3b-3779a55451c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:33.741844 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 774/1559] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:33.743156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f481c65-89c0-42e6-952a-2d35ab703343 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.743436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f481c65-89c0-42e6-952a-2d35ab703343 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.743716 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f481c65-89c0-42e6-952a-2d35ab703343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.763896 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-4f481c65-89c0-42e6-952a-2d35ab703343 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:33.765112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4f481c65-89c0-42e6-952a-2d35ab703343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:33.773074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 775/1560] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:33.774315 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a25be-b9cc-4d14-b257-bb09c38b1257 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.774607 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a25be-b9cc-4d14-b257-bb09c38b1257 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.774926 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a25be-b9cc-4d14-b257-bb09c38b1257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.793385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-547a25be-b9cc-4d14-b257-bb09c38b1257 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:33.794627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-547a25be-b9cc-4d14-b257-bb09c38b1257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:33.803371 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 776/1561] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:33.804701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17370e08-4242-4c2e-b6f1-79978e517fc3 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.804992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17370e08-4242-4c2e-b6f1-79978e517fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.805271 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17370e08-4242-4c2e-b6f1-79978e517fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.826470 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-17370e08-4242-4c2e-b6f1-79978e517fc3 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:33.827594 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-17370e08-4242-4c2e-b6f1-79978e517fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:33.836130 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 777/1562] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:33.836552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17370e08-4242-4c2e-b6f1-79978e517fc3 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:33.847450 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 786/1563] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:31:33 2022] POST /identity/v3/projects => generated 393 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:33.848660 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17370e08-4242-4c2e-b6f1-79978e517fc3 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:33.848918 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0205d-e7ac-477b-8584-f4292cc17662 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:33.864758 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:33.918963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0205d-e7ac-477b-8584-f4292cc17662 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:33.920925 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde32f78-c981-40fb-b007-0395233f8024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:33.922019 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde32f78-c981-40fb-b007-0395233f8024 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.922520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde32f78-c981-40fb-b007-0395233f8024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.923607 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde32f78-c981-40fb-b007-0395233f8024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.924137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bee89-fc8f-40d5-bbb3-777235690c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:33.924289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde32f78-c981-40fb-b007-0395233f8024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:33.924873 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bee89-fc8f-40d5-bbb3-777235690c2e admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.925162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bee89-fc8f-40d5-bbb3-777235690c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.925448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bee89-fc8f-40d5-bbb3-777235690c2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.926737 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bee89-fc8f-40d5-bbb3-777235690c2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:33.927066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde32f78-c981-40fb-b007-0395233f8024 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:33.928756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bee89-fc8f-40d5-bbb3-777235690c2e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:33.941144 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 778/1564] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:33.945824 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b5519-a299-4af2-9aab-1d556dfbbfca None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.946608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b5519-a299-4af2-9aab-1d556dfbbfca None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.947088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b5519-a299-4af2-9aab-1d556dfbbfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:33.967685 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-4e6b5519-a299-4af2-9aab-1d556dfbbfca None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:33.969858 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4e6b5519-a299-4af2-9aab-1d556dfbbfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:33.983815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 779/1565] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:33.983815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a84a-0a8f-41e2-9072-5c5d1dea5a5e None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:33.983815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a84a-0a8f-41e2-9072-5c5d1dea5a5e None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:33.983815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a84a-0a8f-41e2-9072-5c5d1dea5a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.001723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-f080a84a-0a8f-41e2-9072-5c5d1dea5a5e None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:34.003043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f080a84a-0a8f-41e2-9072-5c5d1dea5a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:34.011517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 780/1566] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:34.014989 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 787/1567] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:31:33 2022] POST /identity/v3/projects => generated 371 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:34.014989 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d617391-ce95-4170-a745-9ac18db96d24 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:34.014989 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d617391-ce95-4170-a745-9ac18db96d24 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:34.014989 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d617391-ce95-4170-a745-9ac18db96d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.015389 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080a84a-0a8f-41e2-9072-5c5d1dea5a5e None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:34.026933 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080a84a-0a8f-41e2-9072-5c5d1dea5a5e None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:34.033481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-5d617391-ce95-4170-a745-9ac18db96d24 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:34.034722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-5d617391-ce95-4170-a745-9ac18db96d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:34.088994 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30effcec-f6a0-40aa-aa8f-432deffc241d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:34.089973 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30effcec-f6a0-40aa-aa8f-432deffc241d admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:34.090316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30effcec-f6a0-40aa-aa8f-432deffc241d admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:34.090744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30effcec-f6a0-40aa-aa8f-432deffc241d admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.091580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30effcec-f6a0-40aa-aa8f-432deffc241d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:31:34.092176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30effcec-f6a0-40aa-aa8f-432deffc241d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:34.095297 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30effcec-f6a0-40aa-aa8f-432deffc241d admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:34.108958 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 788/1568] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:34.108958 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b7108-aa1c-44a7-85a4-e1b4a1a4d705 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:34.108958 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b7108-aa1c-44a7-85a4-e1b4a1a4d705 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:34.108958 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b7108-aa1c-44a7-85a4-e1b4a1a4d705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.113748 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 781/1569] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:31:34.114763 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30effcec-f6a0-40aa-aa8f-432deffc241d admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:34.123171 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:34.126972 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30effcec-f6a0-40aa-aa8f-432deffc241d admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:34.129912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-4e1b7108-aa1c-44a7-85a4-e1b4a1a4d705 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:34.131214 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-4e1b7108-aa1c-44a7-85a4-e1b4a1a4d705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:34.132992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934aeca-b410-4d52-a45d-b02ce7342589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:34.133720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934aeca-b410-4d52-a45d-b02ce7342589 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:34.134125 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934aeca-b410-4d52-a45d-b02ce7342589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:34.134364 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934aeca-b410-4d52-a45d-b02ce7342589 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.136751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934aeca-b410-4d52-a45d-b02ce7342589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:34.137252 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934aeca-b410-4d52-a45d-b02ce7342589 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:34.139673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 789/1570] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:34.141128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b7108-aa1c-44a7-85a4-e1b4a1a4d705 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:34.151981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b7108-aa1c-44a7-85a4-e1b4a1a4d705 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:34.156593 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2bceb-bb5c-476a-8a6d-3c35c82ec477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:34.157282 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2bceb-bb5c-476a-8a6d-3c35c82ec477 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:34.157565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2bceb-bb5c-476a-8a6d-3c35c82ec477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:34.157847 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2bceb-bb5c-476a-8a6d-3c35c82ec477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.158626 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2bceb-bb5c-476a-8a6d-3c35c82ec477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:34.160232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2bceb-bb5c-476a-8a6d-3c35c82ec477 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:34.171524 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 790/1571] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:34.172343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2bceb-bb5c-476a-8a6d-3c35c82ec477 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:34.183526 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2bceb-bb5c-476a-8a6d-3c35c82ec477 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:34.188482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31907bf-5bc9-4f07-b315-a955a08a7f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:34.189139 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31907bf-5bc9-4f07-b315-a955a08a7f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:34.189419 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31907bf-5bc9-4f07-b315-a955a08a7f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:34.189692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31907bf-5bc9-4f07-b315-a955a08a7f07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.190459 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31907bf-5bc9-4f07-b315-a955a08a7f07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:31:34.190763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31907bf-5bc9-4f07-b315-a955a08a7f07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:34.192415 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31907bf-5bc9-4f07-b315-a955a08a7f07 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:34.205391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 791/1572] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:31:34.206470 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31907bf-5bc9-4f07-b315-a955a08a7f07 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:34.217463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31907bf-5bc9-4f07-b315-a955a08a7f07 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:34.222680 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ea902-f405-49f6-8b72-1f7874d564c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:34.223341 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ea902-f405-49f6-8b72-1f7874d564c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:34.223625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ea902-f405-49f6-8b72-1f7874d564c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:34.223905 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ea902-f405-49f6-8b72-1f7874d564c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.224596 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ea902-f405-49f6-8b72-1f7874d564c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:34.226540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ea902-f405-49f6-8b72-1f7874d564c2 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:34.258194 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 782/1573] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:31:34 2022] POST /identity/v3/users => generated 297 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:34.261797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934aeca-b410-4d52-a45d-b02ce7342589 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:34.269915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934aeca-b410-4d52-a45d-b02ce7342589 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:34.275465 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71491f3-b959-4ad2-a028-5993dcb6ec63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:34.276865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71491f3-b959-4ad2-a028-5993dcb6ec63 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:34.277214 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71491f3-b959-4ad2-a028-5993dcb6ec63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:34.277609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71491f3-b959-4ad2-a028-5993dcb6ec63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.278417 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71491f3-b959-4ad2-a028-5993dcb6ec63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:31:34.284634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71491f3-b959-4ad2-a028-5993dcb6ec63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:34.284634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71491f3-b959-4ad2-a028-5993dcb6ec63 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:34.294512 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 783/1574] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:31:34.295335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71491f3-b959-4ad2-a028-5993dcb6ec63 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:34.305417 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71491f3-b959-4ad2-a028-5993dcb6ec63 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:34.311015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967609d-4b22-470b-89fc-d52e651fc4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:34.311556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967609d-4b22-470b-89fc-d52e651fc4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:34.311832 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967609d-4b22-470b-89fc-d52e651fc4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:34.312111 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967609d-4b22-470b-89fc-d52e651fc4b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.312840 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967609d-4b22-470b-89fc-d52e651fc4b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:31:34.313144 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967609d-4b22-470b-89fc-d52e651fc4b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:34.314823 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967609d-4b22-470b-89fc-d52e651fc4b9 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:34.327183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 784/1575] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:31:34.327994 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967609d-4b22-470b-89fc-d52e651fc4b9 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:34.340948 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967609d-4b22-470b-89fc-d52e651fc4b9 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:34.345794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446048c-22e6-455f-9fe7-6018977217b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:34.346528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446048c-22e6-455f-9fe7-6018977217b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:34.346821 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446048c-22e6-455f-9fe7-6018977217b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:34.347099 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446048c-22e6-455f-9fe7-6018977217b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.347797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446048c-22e6-455f-9fe7-6018977217b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:34.349729 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446048c-22e6-455f-9fe7-6018977217b0 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:34.354990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 792/1576] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:31:34 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:34.355896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ea902-f405-49f6-8b72-1f7874d564c2 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:34.368835 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ea902-f405-49f6-8b72-1f7874d564c2 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:34.374354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b32f8a-093c-4f9d-b7ea-e45feb72497f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:34.375076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b32f8a-093c-4f9d-b7ea-e45feb72497f admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:34.375364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b32f8a-093c-4f9d-b7ea-e45feb72497f admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:34.375651 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b32f8a-093c-4f9d-b7ea-e45feb72497f admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.376445 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b32f8a-093c-4f9d-b7ea-e45feb72497f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:31:34.376744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b32f8a-093c-4f9d-b7ea-e45feb72497f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:34.378449 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b32f8a-093c-4f9d-b7ea-e45feb72497f admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:34.393267 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 793/1577] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:31:34.395202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b32f8a-093c-4f9d-b7ea-e45feb72497f admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:34.404700 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b32f8a-093c-4f9d-b7ea-e45feb72497f admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:34.410405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121a7da-a3dc-4542-906f-d9ece2daacf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:34.411777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121a7da-a3dc-4542-906f-d9ece2daacf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:34.412185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121a7da-a3dc-4542-906f-d9ece2daacf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:34.412466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121a7da-a3dc-4542-906f-d9ece2daacf3 admin admin] PATH_INFO: `/v3/projects/aa49d7002cf441348cf3166e64b1bc87/users/8e75c180bae74be0b2ccad079f92d32b/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.428865 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0121a7da-a3dc-4542-906f-d9ece2daacf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa49d7002cf441348cf3166e64b1bc87, user_id=8e75c180bae74be0b2ccad079f92d32b, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:34.430216 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: WARNING py.warnings [None req-0121a7da-a3dc-4542-906f-d9ece2daacf3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:31:34.430216 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: warnings.warn(msg) Jul 20 15:31:34.430216 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: Jul 20 15:31:34.433156 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0121a7da-a3dc-4542-906f-d9ece2daacf3 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:34.466348 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 785/1578] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:31:34 2022] POST /identity/v3/projects => generated 387 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:34.466348 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446048c-22e6-455f-9fe7-6018977217b0 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:34.497113 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:34.529517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446048c-22e6-455f-9fe7-6018977217b0 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:34.534399 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf09be4-0e39-4ecf-88fb-eba300edf944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:34.535067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf09be4-0e39-4ecf-88fb-eba300edf944 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:34.535353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf09be4-0e39-4ecf-88fb-eba300edf944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:34.535630 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf09be4-0e39-4ecf-88fb-eba300edf944 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.536329 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf09be4-0e39-4ecf-88fb-eba300edf944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:34.538434 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf09be4-0e39-4ecf-88fb-eba300edf944 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:34.559082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 794/1579] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:34 2022] PUT /identity/v3/projects/aa49d7002cf441348cf3166e64b1bc87/users/8e75c180bae74be0b2ccad079f92d32b/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:34.561452 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121a7da-a3dc-4542-906f-d9ece2daacf3 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:34.633461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121a7da-a3dc-4542-906f-d9ece2daacf3 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:34.638342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e350456-5c85-4501-90e6-b2fda7a2d911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:34.639028 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e350456-5c85-4501-90e6-b2fda7a2d911 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:34.639310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e350456-5c85-4501-90e6-b2fda7a2d911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:34.639587 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e350456-5c85-4501-90e6-b2fda7a2d911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.640466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e350456-5c85-4501-90e6-b2fda7a2d911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:34.641974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e350456-5c85-4501-90e6-b2fda7a2d911 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:34.655401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 786/1580] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:31:34 2022] POST /identity/v3/projects => generated 371 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:34.655648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 795/1581] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:34.655753 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf09be4-0e39-4ecf-88fb-eba300edf944 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:34.655856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e350456-5c85-4501-90e6-b2fda7a2d911 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:34.670880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e350456-5c85-4501-90e6-b2fda7a2d911 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:34.730882 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf09be4-0e39-4ecf-88fb-eba300edf944 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:34.737316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063a265-7424-4052-bd8f-4dfd46525f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:34.738151 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063a265-7424-4052-bd8f-4dfd46525f57 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:34.738672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063a265-7424-4052-bd8f-4dfd46525f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:34.739077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063a265-7424-4052-bd8f-4dfd46525f57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.739917 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063a265-7424-4052-bd8f-4dfd46525f57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:34.742170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063a265-7424-4052-bd8f-4dfd46525f57 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:34.760111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7696a80-d0bd-42ac-8f3c-d277d47e2fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:34.760837 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7696a80-d0bd-42ac-8f3c-d277d47e2fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:34.761125 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7696a80-d0bd-42ac-8f3c-d277d47e2fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:34.761406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7696a80-d0bd-42ac-8f3c-d277d47e2fa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.762163 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7696a80-d0bd-42ac-8f3c-d277d47e2fa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:34.764175 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7696a80-d0bd-42ac-8f3c-d277d47e2fa9 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:34.884352 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 787/1582] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:31:34 2022] POST /identity/v3/projects => generated 379 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:34.884352 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063a265-7424-4052-bd8f-4dfd46525f57 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:34.947646 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063a265-7424-4052-bd8f-4dfd46525f57 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:34.953034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d13db-8095-4f9a-928c-1eab1bb1aca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:34.953709 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d13db-8095-4f9a-928c-1eab1bb1aca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:34.953992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d13db-8095-4f9a-928c-1eab1bb1aca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:34.954304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d13db-8095-4f9a-928c-1eab1bb1aca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:34.955020 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d13db-8095-4f9a-928c-1eab1bb1aca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:34.956644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d13db-8095-4f9a-928c-1eab1bb1aca6 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:34.971830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 788/1583] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:34.974991 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d13db-8095-4f9a-928c-1eab1bb1aca6 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:34.979785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 796/1584] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:31:34 2022] POST /identity/v3/projects => generated 383 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:34.980912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7696a80-d0bd-42ac-8f3c-d277d47e2fa9 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:34.987053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d13db-8095-4f9a-928c-1eab1bb1aca6 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:34.989360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:35.025324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:35.054105 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7696a80-d0bd-42ac-8f3c-d277d47e2fa9 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:35.057746 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffc7d8-cf82-45e1-80ac-fd00a2e3d7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:35.058474 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffc7d8-cf82-45e1-80ac-fd00a2e3d7d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:35.058762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffc7d8-cf82-45e1-80ac-fd00a2e3d7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:35.058881 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ed616-1e38-4bac-9849-bf2098aa955a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:35.059066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffc7d8-cf82-45e1-80ac-fd00a2e3d7d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:35.059590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ed616-1e38-4bac-9849-bf2098aa955a admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:35.059814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fffc7d8-cf82-45e1-80ac-fd00a2e3d7d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:35.059921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ed616-1e38-4bac-9849-bf2098aa955a admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:35.060162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ed616-1e38-4bac-9849-bf2098aa955a admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:35.060868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ed616-1e38-4bac-9849-bf2098aa955a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:35.061735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fffc7d8-cf82-45e1-80ac-fd00a2e3d7d1 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:35.062801 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ed616-1e38-4bac-9849-bf2098aa955a admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:35.280153 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 797/1585] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:31:34 2022] POST /identity/v3/users => generated 294 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:35.289372 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cf0a3-84ef-4503-9616-a12d4ab7f44b None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:35.289372 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cf0a3-84ef-4503-9616-a12d4ab7f44b None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:35.289372 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cf0a3-84ef-4503-9616-a12d4ab7f44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:35.300254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 789/1586] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:31:34 2022] POST /identity/v3/projects => generated 367 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:35.301709 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffc7d8-cf82-45e1-80ac-fd00a2e3d7d1 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:35.306771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-b65cf0a3-84ef-4503-9616-a12d4ab7f44b None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:35.308570 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b65cf0a3-84ef-4503-9616-a12d4ab7f44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:35.370177 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffc7d8-cf82-45e1-80ac-fd00a2e3d7d1 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:35.374970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f61cc-c943-4ecd-bc65-6b0ecb370220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:35.375658 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f61cc-c943-4ecd-bc65-6b0ecb370220 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:35.375944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f61cc-c943-4ecd-bc65-6b0ecb370220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:35.376237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f61cc-c943-4ecd-bc65-6b0ecb370220 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:35.376936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f61cc-c943-4ecd-bc65-6b0ecb370220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:35.378886 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f61cc-c943-4ecd-bc65-6b0ecb370220 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:35.379846 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 798/1587] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:35.383431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65cf0a3-84ef-4503-9616-a12d4ab7f44b None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:35.396515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65cf0a3-84ef-4503-9616-a12d4ab7f44b None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:35.401948 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832bf7e-40a9-4045-ada1-55304f92e159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:35.402743 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832bf7e-40a9-4045-ada1-55304f92e159 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:35.407969 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832bf7e-40a9-4045-ada1-55304f92e159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:35.407969 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832bf7e-40a9-4045-ada1-55304f92e159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:35.407969 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e832bf7e-40a9-4045-ada1-55304f92e159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:35.407969 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e832bf7e-40a9-4045-ada1-55304f92e159 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:35.416043 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 799/1588] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:35.416940 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832bf7e-40a9-4045-ada1-55304f92e159 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:35.426649 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832bf7e-40a9-4045-ada1-55304f92e159 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:35.432100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c8ecc-645a-4bda-8fd7-93afa0b7e11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:35.432747 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c8ecc-645a-4bda-8fd7-93afa0b7e11a admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:35.433033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c8ecc-645a-4bda-8fd7-93afa0b7e11a admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:35.433312 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c8ecc-645a-4bda-8fd7-93afa0b7e11a admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:35.433999 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c8ecc-645a-4bda-8fd7-93afa0b7e11a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:35.435950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c8ecc-645a-4bda-8fd7-93afa0b7e11a admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:35.540571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 790/1589] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:31:35 2022] POST /identity/v3/users => generated 286 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:35.540571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f61cc-c943-4ecd-bc65-6b0ecb370220 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:35.548463 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f61cc-c943-4ecd-bc65-6b0ecb370220 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:35.554168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823ce39-aa7d-406d-92a4-ffe97ac22520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:35.554504 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823ce39-aa7d-406d-92a4-ffe97ac22520 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:35.554796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823ce39-aa7d-406d-92a4-ffe97ac22520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:35.555076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823ce39-aa7d-406d-92a4-ffe97ac22520 admin admin] PATH_INFO: `/v3/projects/6afe2cf697fd469d97865059a2700ed6/users/69c0f9504d6c434a99cc2671c483f689/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:35.570447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823ce39-aa7d-406d-92a4-ffe97ac22520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6afe2cf697fd469d97865059a2700ed6, user_id=69c0f9504d6c434a99cc2671c483f689, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:35.573604 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823ce39-aa7d-406d-92a4-ffe97ac22520 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:35.598391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 800/1590] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:31:35 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:35.598391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c8ecc-645a-4bda-8fd7-93afa0b7e11a admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:35.608129 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c8ecc-645a-4bda-8fd7-93afa0b7e11a admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:35.613115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561eeeff-e9a9-485c-9572-f4a96b1b47b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:35.613855 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561eeeff-e9a9-485c-9572-f4a96b1b47b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:35.614144 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561eeeff-e9a9-485c-9572-f4a96b1b47b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:35.614624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561eeeff-e9a9-485c-9572-f4a96b1b47b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:35.615578 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561eeeff-e9a9-485c-9572-f4a96b1b47b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:35.617640 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561eeeff-e9a9-485c-9572-f4a96b1b47b3 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:35.691150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 791/1591] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:35 2022] PUT /identity/v3/projects/6afe2cf697fd469d97865059a2700ed6/users/69c0f9504d6c434a99cc2671c483f689/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:35.692265 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823ce39-aa7d-406d-92a4-ffe97ac22520 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:35.757686 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823ce39-aa7d-406d-92a4-ffe97ac22520 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:35.763089 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f7edb-9c5e-4293-baae-13a202b00cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:35.763867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f7edb-9c5e-4293-baae-13a202b00cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:35.764179 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f7edb-9c5e-4293-baae-13a202b00cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:35.764468 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f7edb-9c5e-4293-baae-13a202b00cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:35.765206 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f7edb-9c5e-4293-baae-13a202b00cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:35.766929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f7edb-9c5e-4293-baae-13a202b00cd8 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:35.777105 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 792/1592] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:35.777934 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f7edb-9c5e-4293-baae-13a202b00cd8 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:35.785986 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 801/1593] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:31:35 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:35.786853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561eeeff-e9a9-485c-9572-f4a96b1b47b3 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:35.787764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f7edb-9c5e-4293-baae-13a202b00cd8 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:35.792573 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc84f3-4e7e-42da-a0bb-e1961200bd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:35.793191 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc84f3-4e7e-42da-a0bb-e1961200bd6c admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:35.793481 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc84f3-4e7e-42da-a0bb-e1961200bd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:35.793765 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc84f3-4e7e-42da-a0bb-e1961200bd6c admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:35.794510 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc84f3-4e7e-42da-a0bb-e1961200bd6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:35.796463 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc84f3-4e7e-42da-a0bb-e1961200bd6c admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:35.850359 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:35.856005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561eeeff-e9a9-485c-9572-f4a96b1b47b3 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:35.907457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402268ff-8bdd-4237-ab6f-a35599bd4e45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:35.908165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402268ff-8bdd-4237-ab6f-a35599bd4e45 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:35.908497 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402268ff-8bdd-4237-ab6f-a35599bd4e45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:35.908779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402268ff-8bdd-4237-ab6f-a35599bd4e45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:35.918550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402268ff-8bdd-4237-ab6f-a35599bd4e45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:35.919993 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402268ff-8bdd-4237-ab6f-a35599bd4e45 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:35.930461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 802/1594] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:35.931276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402268ff-8bdd-4237-ab6f-a35599bd4e45 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:35.941420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402268ff-8bdd-4237-ab6f-a35599bd4e45 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:35.946899 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784fbc5-b229-40f3-a456-22cef197d947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:35.947793 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784fbc5-b229-40f3-a456-22cef197d947 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:35.948574 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784fbc5-b229-40f3-a456-22cef197d947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:35.948859 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784fbc5-b229-40f3-a456-22cef197d947 admin admin] PATH_INFO: `/v3/projects/61872ef687074f5791a2cc46732bebdf/users/ebbcfe6341a3486d93e12c09f83954dc/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:35.955448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 793/1595] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:31:35 2022] POST /identity/v3/users => generated 284 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:35.955448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc84f3-4e7e-42da-a0bb-e1961200bd6c admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:35.965192 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc84f3-4e7e-42da-a0bb-e1961200bd6c admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:35.968316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37577cb3-3472-470a-a2f6-09c6bee233fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:35.969028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37577cb3-3472-470a-a2f6-09c6bee233fc admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:35.969308 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37577cb3-3472-470a-a2f6-09c6bee233fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:35.969587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37577cb3-3472-470a-a2f6-09c6bee233fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:35.970343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37577cb3-3472-470a-a2f6-09c6bee233fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:35.971952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37577cb3-3472-470a-a2f6-09c6bee233fc admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:35.977975 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784fbc5-b229-40f3-a456-22cef197d947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61872ef687074f5791a2cc46732bebdf, user_id=ebbcfe6341a3486d93e12c09f83954dc, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:35.980889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784fbc5-b229-40f3-a456-22cef197d947 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:35.983032 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 794/1596] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:35.983870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37577cb3-3472-470a-a2f6-09c6bee233fc admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:35.993777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:35.999728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37577cb3-3472-470a-a2f6-09c6bee233fc admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:36.005196 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cb1d5-5add-4ea0-b71d-19780ce2877c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:36.005859 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cb1d5-5add-4ea0-b71d-19780ce2877c admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.006141 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cb1d5-5add-4ea0-b71d-19780ce2877c admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.006446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cb1d5-5add-4ea0-b71d-19780ce2877c admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.007171 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030cb1d5-5add-4ea0-b71d-19780ce2877c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:36.008840 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030cb1d5-5add-4ea0-b71d-19780ce2877c admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:36.019873 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 795/1597] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:36.020670 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cb1d5-5add-4ea0-b71d-19780ce2877c admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:36.030189 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cb1d5-5add-4ea0-b71d-19780ce2877c admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:36.035076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3a07a-8d0f-41ad-8e76-a89f17322f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:36.035726 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3a07a-8d0f-41ad-8e76-a89f17322f52 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.036010 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3a07a-8d0f-41ad-8e76-a89f17322f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.036285 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3a07a-8d0f-41ad-8e76-a89f17322f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.036999 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3a07a-8d0f-41ad-8e76-a89f17322f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:36.038597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3a07a-8d0f-41ad-8e76-a89f17322f52 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:36.048839 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 796/1598] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:36.049565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3a07a-8d0f-41ad-8e76-a89f17322f52 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:36.060189 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3a07a-8d0f-41ad-8e76-a89f17322f52 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:36.064828 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00659e7-1bea-486f-82c9-84898d78996a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:36.065495 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00659e7-1bea-486f-82c9-84898d78996a admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.065761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00659e7-1bea-486f-82c9-84898d78996a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.066159 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00659e7-1bea-486f-82c9-84898d78996a admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.066918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00659e7-1bea-486f-82c9-84898d78996a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:36.083131 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00659e7-1bea-486f-82c9-84898d78996a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:36.083131 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 797/1599] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:36.083131 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00659e7-1bea-486f-82c9-84898d78996a admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:36.102346 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00659e7-1bea-486f-82c9-84898d78996a admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:36.108245 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf860606-7734-4ca2-8574-aa9556e65c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:36.108923 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf860606-7734-4ca2-8574-aa9556e65c08 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.115059 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf860606-7734-4ca2-8574-aa9556e65c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.115059 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf860606-7734-4ca2-8574-aa9556e65c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.115059 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf860606-7734-4ca2-8574-aa9556e65c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:36.115059 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf860606-7734-4ca2-8574-aa9556e65c08 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:36.123279 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 798/1600] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:36.124126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf860606-7734-4ca2-8574-aa9556e65c08 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:36.135666 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf860606-7734-4ca2-8574-aa9556e65c08 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:36.144095 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5aca1-ad9b-4b27-800b-5eef2e5590c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:36.144783 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5aca1-ad9b-4b27-800b-5eef2e5590c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.145073 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5aca1-ad9b-4b27-800b-5eef2e5590c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.145357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5aca1-ad9b-4b27-800b-5eef2e5590c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.146324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d5aca1-ad9b-4b27-800b-5eef2e5590c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:36.147805 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d5aca1-ad9b-4b27-800b-5eef2e5590c5 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:36.159290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 803/1601] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:35 2022] PUT /identity/v3/projects/61872ef687074f5791a2cc46732bebdf/users/ebbcfe6341a3486d93e12c09f83954dc/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:36.160797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784fbc5-b229-40f3-a456-22cef197d947 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:36.165814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 799/1602] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:36.167054 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5aca1-ad9b-4b27-800b-5eef2e5590c5 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:36.177545 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5aca1-ad9b-4b27-800b-5eef2e5590c5 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:36.249485 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784fbc5-b229-40f3-a456-22cef197d947 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:36.256089 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b2e0d-2007-4369-96ac-fb738f2e7f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:36.256654 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d75e1-50a1-4ce5-a807-01dd490de784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:36.256848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b2e0d-2007-4369-96ac-fb738f2e7f98 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.257138 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b2e0d-2007-4369-96ac-fb738f2e7f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.257320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d75e1-50a1-4ce5-a807-01dd490de784 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.257435 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b2e0d-2007-4369-96ac-fb738f2e7f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.257602 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d75e1-50a1-4ce5-a807-01dd490de784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.257879 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d75e1-50a1-4ce5-a807-01dd490de784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.258249 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b2e0d-2007-4369-96ac-fb738f2e7f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:36.258641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d75e1-50a1-4ce5-a807-01dd490de784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:36.260257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d75e1-50a1-4ce5-a807-01dd490de784 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:36.271170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b2e0d-2007-4369-96ac-fb738f2e7f98 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:36.272386 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 800/1603] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:36.273242 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d75e1-50a1-4ce5-a807-01dd490de784 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:36.283807 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d75e1-50a1-4ce5-a807-01dd490de784 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:36.286593 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 804/1604] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:36.287499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b2e0d-2007-4369-96ac-fb738f2e7f98 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:36.292588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69679f37-0f7f-479d-a095-1a8f979a1a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:36.294561 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69679f37-0f7f-479d-a095-1a8f979a1a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.295498 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69679f37-0f7f-479d-a095-1a8f979a1a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.295780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69679f37-0f7f-479d-a095-1a8f979a1a43 admin admin] PATH_INFO: `/v3/projects/6afe2cf697fd469d97865059a2700ed6/users/69c0f9504d6c434a99cc2671c483f689/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.298055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69679f37-0f7f-479d-a095-1a8f979a1a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6afe2cf697fd469d97865059a2700ed6, user_id=69c0f9504d6c434a99cc2671c483f689, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:36.298573 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b2e0d-2007-4369-96ac-fb738f2e7f98 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:36.300857 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69679f37-0f7f-479d-a095-1a8f979a1a43 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:36.303311 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe5348-d241-47a1-9a78-8116d00bc7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:36.304212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe5348-d241-47a1-9a78-8116d00bc7ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.304497 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe5348-d241-47a1-9a78-8116d00bc7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.304764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe5348-d241-47a1-9a78-8116d00bc7ce admin admin] PATH_INFO: `/v3/projects/4446d4a6f4e64ccb8bfd5b3fe6f2127a/users/6498ef5c517c4fa4abed90e430936ded/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.312957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 801/1605] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:36 2022] PUT /identity/v3/projects/6afe2cf697fd469d97865059a2700ed6/users/69c0f9504d6c434a99cc2671c483f689/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:36.313808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69679f37-0f7f-479d-a095-1a8f979a1a43 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:36.328693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe5348-d241-47a1-9a78-8116d00bc7ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=4446d4a6f4e64ccb8bfd5b3fe6f2127a, user_id=6498ef5c517c4fa4abed90e430936ded, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:36.331807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe5348-d241-47a1-9a78-8116d00bc7ce admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:36.399269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69679f37-0f7f-479d-a095-1a8f979a1a43 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:36.405626 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff96933-0003-4631-94ab-f1f303d8923e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:36.406338 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff96933-0003-4631-94ab-f1f303d8923e admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.406634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff96933-0003-4631-94ab-f1f303d8923e admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.406915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff96933-0003-4631-94ab-f1f303d8923e admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.407722 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff96933-0003-4631-94ab-f1f303d8923e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:36.409457 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff96933-0003-4631-94ab-f1f303d8923e admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:36.421573 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 802/1606] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:36.425073 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 805/1607] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:36 2022] PUT /identity/v3/projects/4446d4a6f4e64ccb8bfd5b3fe6f2127a/users/6498ef5c517c4fa4abed90e430936ded/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:36.425264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29db2ef-7970-4eb2-95df-f9644e4cbf3e None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.425430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe5348-d241-47a1-9a78-8116d00bc7ce admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:36.425611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29db2ef-7970-4eb2-95df-f9644e4cbf3e None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.425887 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29db2ef-7970-4eb2-95df-f9644e4cbf3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.454687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-a29db2ef-7970-4eb2-95df-f9644e4cbf3e None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:36.457522 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a29db2ef-7970-4eb2-95df-f9644e4cbf3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:36.485216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:36.511859 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:36.526592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe5348-d241-47a1-9a78-8116d00bc7ce admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:36.534440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda98a6c-c73c-4c32-a2b5-4553d3bc1d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:36.535123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda98a6c-c73c-4c32-a2b5-4553d3bc1d04 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.535404 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda98a6c-c73c-4c32-a2b5-4553d3bc1d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.535677 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda98a6c-c73c-4c32-a2b5-4553d3bc1d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.536405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda98a6c-c73c-4c32-a2b5-4553d3bc1d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:36.538009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda98a6c-c73c-4c32-a2b5-4553d3bc1d04 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:36.549102 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 806/1608] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:36.552080 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda98a6c-c73c-4c32-a2b5-4553d3bc1d04 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:36.562889 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 803/1609] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:36.563920 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29db2ef-7970-4eb2-95df-f9644e4cbf3e None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:36.564264 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda98a6c-c73c-4c32-a2b5-4553d3bc1d04 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:36.569834 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256e808-9594-4cac-b540-de071e86f24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:36.570896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256e808-9594-4cac-b540-de071e86f24c admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.571255 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256e808-9594-4cac-b540-de071e86f24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.571850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256e808-9594-4cac-b540-de071e86f24c admin admin] PATH_INFO: `/v3/projects/80caade751834762862c102c208a34e1/users/48ce3bfba3724b07a657774662b8d6f3/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.581092 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29db2ef-7970-4eb2-95df-f9644e4cbf3e None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:36.586805 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65604a4-92bf-47cd-84e2-949e4e4d303f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:36.587727 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65604a4-92bf-47cd-84e2-949e4e4d303f admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.588042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65604a4-92bf-47cd-84e2-949e4e4d303f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.588358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65604a4-92bf-47cd-84e2-949e4e4d303f admin admin] PATH_INFO: `/v3/projects/35e8192d825741039ebf0f25fc22511e/users/336c572cd6154216a4ca6468fb6876fe/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.592366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256e808-9594-4cac-b540-de071e86f24c admin admin] RBAC: Authorizing `identity:create_grant(project_id=80caade751834762862c102c208a34e1, user_id=48ce3bfba3724b07a657774662b8d6f3, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:36.595472 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256e808-9594-4cac-b540-de071e86f24c admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:36.605816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65604a4-92bf-47cd-84e2-949e4e4d303f admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e8192d825741039ebf0f25fc22511e, user_id=336c572cd6154216a4ca6468fb6876fe, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:36.609180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65604a4-92bf-47cd-84e2-949e4e4d303f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:36.702600 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 807/1610] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:36 2022] PUT /identity/v3/projects/80caade751834762862c102c208a34e1/users/48ce3bfba3724b07a657774662b8d6f3/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:36.707495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953eeed-634f-4115-906d-ff8cdc6a021d None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.707495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953eeed-634f-4115-906d-ff8cdc6a021d None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.707495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953eeed-634f-4115-906d-ff8cdc6a021d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.742868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-9953eeed-634f-4115-906d-ff8cdc6a021d None None] MFA Rules not processed for user `69c0f9504d6c434a99cc2671c483f689`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:36.744439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9953eeed-634f-4115-906d-ff8cdc6a021d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:36.758135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 804/1611] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:36 2022] PUT /identity/v3/projects/35e8192d825741039ebf0f25fc22511e/users/336c572cd6154216a4ca6468fb6876fe/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:36.759028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65604a4-92bf-47cd-84e2-949e4e4d303f admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:36.847811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65604a4-92bf-47cd-84e2-949e4e4d303f admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:36.852889 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c89d7-3dc4-4f2b-b461-3f4099b958e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:36.853831 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c89d7-3dc4-4f2b-b461-3f4099b958e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.858242 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c89d7-3dc4-4f2b-b461-3f4099b958e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.858242 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c89d7-3dc4-4f2b-b461-3f4099b958e1 admin admin] PATH_INFO: `/v3/projects/99f5f8dca0134df7b339987c309d2382/users/ca0d143cb0154a0d854bdb9509216395/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.873836 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 808/1612] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:36.875393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92a7ff-a081-4ece-a138-4df13658c453 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.875709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92a7ff-a081-4ece-a138-4df13658c453 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.875968 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92a7ff-a081-4ece-a138-4df13658c453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.876639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c89d7-3dc4-4f2b-b461-3f4099b958e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99f5f8dca0134df7b339987c309d2382, user_id=ca0d143cb0154a0d854bdb9509216395, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:36.879682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c89d7-3dc4-4f2b-b461-3f4099b958e1 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:36.897719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-db92a7ff-a081-4ece-a138-4df13658c453 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:36.899030 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-db92a7ff-a081-4ece-a138-4df13658c453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:36.907848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 809/1613] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:36.908701 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92a7ff-a081-4ece-a138-4df13658c453 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:36.921450 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92a7ff-a081-4ece-a138-4df13658c453 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:36.926565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d746d-5bb8-4fc6-b1ba-a37db275e11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:36.933143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d746d-5bb8-4fc6-b1ba-a37db275e11a admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:36.933143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d746d-5bb8-4fc6-b1ba-a37db275e11a admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:36.933143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d746d-5bb8-4fc6-b1ba-a37db275e11a admin admin] PATH_INFO: `/v3/projects/493fb99c808d45e186749e61845c48dc/users/5140c1159e6f46959cc44b809074c61a/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:36.949212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d746d-5bb8-4fc6-b1ba-a37db275e11a admin admin] RBAC: Authorizing `identity:create_grant(project_id=493fb99c808d45e186749e61845c48dc, user_id=5140c1159e6f46959cc44b809074c61a, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:36.951159 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d746d-5bb8-4fc6-b1ba-a37db275e11a admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:37.012668 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 805/1614] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:36 2022] PUT /identity/v3/projects/99f5f8dca0134df7b339987c309d2382/users/ca0d143cb0154a0d854bdb9509216395/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:37.013863 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c89d7-3dc4-4f2b-b461-3f4099b958e1 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:37.015271 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-894c89d7-3dc4-4f2b-b461-3f4099b958e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:37.093628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c89d7-3dc4-4f2b-b461-3f4099b958e1 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:37.107444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 810/1615] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:36 2022] PUT /identity/v3/projects/493fb99c808d45e186749e61845c48dc/users/5140c1159e6f46959cc44b809074c61a/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:37.107444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99853a-b909-4dc7-ad1c-680c241f07ac None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:37.107444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99853a-b909-4dc7-ad1c-680c241f07ac None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:37.107444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99853a-b909-4dc7-ad1c-680c241f07ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:37.121497 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-ee99853a-b909-4dc7-ad1c-680c241f07ac None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:37.122820 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ee99853a-b909-4dc7-ad1c-680c241f07ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:37.141985 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:37.162010 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc43e44-94b5-41c0-aaf7-0f8326006ac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:37.162716 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc43e44-94b5-41c0-aaf7-0f8326006ac4 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:37.163114 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc43e44-94b5-41c0-aaf7-0f8326006ac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:37.163367 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc43e44-94b5-41c0-aaf7-0f8326006ac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:37.197406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 811/1616] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:37.197406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf1619-2962-438d-be48-0a8470a5b7f5 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:37.197406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf1619-2962-438d-be48-0a8470a5b7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:37.197406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf1619-2962-438d-be48-0a8470a5b7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:37.230499 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-0cdf1619-2962-438d-be48-0a8470a5b7f5 None None] MFA Rules not processed for user `336c572cd6154216a4ca6468fb6876fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:37.231938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0cdf1619-2962-438d-be48-0a8470a5b7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:37.243000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:37.250754 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc43e44-94b5-41c0-aaf7-0f8326006ac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:37.257249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc43e44-94b5-41c0-aaf7-0f8326006ac4 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:37.263055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 806/1617] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:37.263775 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc43e44-94b5-41c0-aaf7-0f8326006ac4 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:37.335945 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 812/1618] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:37.337415 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf1619-2962-438d-be48-0a8470a5b7f5 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:37.348571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc43e44-94b5-41c0-aaf7-0f8326006ac4 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:37.349095 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf1619-2962-438d-be48-0a8470a5b7f5 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:37.352460 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980a74d-4196-42ea-a0c2-d687104e6652 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:37.360895 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a74d-4196-42ea-a0c2-d687104e6652 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:37.360895 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a74d-4196-42ea-a0c2-d687104e6652 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:37.360895 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a74d-4196-42ea-a0c2-d687104e6652 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:37.361320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413cc29-f328-408d-b5c3-26baca405e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:37.361320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413cc29-f328-408d-b5c3-26baca405e58 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:37.361320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413cc29-f328-408d-b5c3-26baca405e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:37.361320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413cc29-f328-408d-b5c3-26baca405e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:37.373107 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413cc29-f328-408d-b5c3-26baca405e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:37.374759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413cc29-f328-408d-b5c3-26baca405e58 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:37.385255 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 813/1619] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:37.386562 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296f702-4fa1-49a6-89db-97f7c6ccb5a6 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:37.386847 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296f702-4fa1-49a6-89db-97f7c6ccb5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:37.387273 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296f702-4fa1-49a6-89db-97f7c6ccb5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:37.424091 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6296f702-4fa1-49a6-89db-97f7c6ccb5a6 None None] MFA Rules not processed for user `ca0d143cb0154a0d854bdb9509216395`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:37.425404 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6296f702-4fa1-49a6-89db-97f7c6ccb5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:37.457888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980a74d-4196-42ea-a0c2-d687104e6652 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:37.459502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980a74d-4196-42ea-a0c2-d687104e6652 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:37.473343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 807/1620] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:37 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:37.474848 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a47809-d68c-4625-bd0c-77076df2ccfa None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:37.475147 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a47809-d68c-4625-bd0c-77076df2ccfa None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:37.475474 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a47809-d68c-4625-bd0c-77076df2ccfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:37.508540 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-13a47809-d68c-4625-bd0c-77076df2ccfa None None] MFA Rules not processed for user `5140c1159e6f46959cc44b809074c61a`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:37.515634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-13a47809-d68c-4625-bd0c-77076df2ccfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:37.519287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 814/1621] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:37.520351 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296f702-4fa1-49a6-89db-97f7c6ccb5a6 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:37.550355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:37.566629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296f702-4fa1-49a6-89db-97f7c6ccb5a6 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:37.570602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d87a6-6dd1-4390-9c60-1dcd1e45b9c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:37.571276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d87a6-6dd1-4390-9c60-1dcd1e45b9c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:37.571565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d87a6-6dd1-4390-9c60-1dcd1e45b9c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:37.572042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d87a6-6dd1-4390-9c60-1dcd1e45b9c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:37.584867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d87a6-6dd1-4390-9c60-1dcd1e45b9c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:37.586455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d87a6-6dd1-4390-9c60-1dcd1e45b9c9 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:37.598443 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 815/1622] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:37.599286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d87a6-6dd1-4390-9c60-1dcd1e45b9c9 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:37.614665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 808/1623] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:37 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:37.616920 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae592603-511f-45aa-bcf0-a8f38d2cd7a9 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:37.616920 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae592603-511f-45aa-bcf0-a8f38d2cd7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:37.616920 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae592603-511f-45aa-bcf0-a8f38d2cd7a9 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:37.619583 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d87a6-6dd1-4390-9c60-1dcd1e45b9c9 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:37.627190 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 809/1624] 10.222.0.78 () {54 vars in 962 bytes} [Wed Jul 20 15:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:31:37.628719 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae592603-511f-45aa-bcf0-a8f38d2cd7a9 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:37.643984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae592603-511f-45aa-bcf0-a8f38d2cd7a9 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:37.649620 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d8643-e42b-48a4-8fe1-99e313bf461c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:37.650419 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d8643-e42b-48a4-8fe1-99e313bf461c admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:37.651509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d8643-e42b-48a4-8fe1-99e313bf461c admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:37.652070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d8643-e42b-48a4-8fe1-99e313bf461c admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:37.653057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25ea99-bbbc-4a6e-bb65-6ec95bc28c04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:37.653196 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d8643-e42b-48a4-8fe1-99e313bf461c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:37.653803 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25ea99-bbbc-4a6e-bb65-6ec95bc28c04 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:37.654225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25ea99-bbbc-4a6e-bb65-6ec95bc28c04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:37.654542 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25ea99-bbbc-4a6e-bb65-6ec95bc28c04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:37.655827 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d8643-e42b-48a4-8fe1-99e313bf461c admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:37.667215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25ea99-bbbc-4a6e-bb65-6ec95bc28c04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:37.668866 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25ea99-bbbc-4a6e-bb65-6ec95bc28c04 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:37.695970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 810/1625] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:37.701046 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671d63b-c84c-4694-ae05-95fde7c61194 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:37.701335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671d63b-c84c-4694-ae05-95fde7c61194 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:37.701631 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671d63b-c84c-4694-ae05-95fde7c61194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:37.763384 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-7671d63b-c84c-4694-ae05-95fde7c61194 None None] MFA Rules not processed for user `a4a24d5a375945779fa626fae25ac237`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:37.765091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-7671d63b-c84c-4694-ae05-95fde7c61194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:37.814942 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 816/1626] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:31:37 2022] POST /identity/v3/projects => generated 373 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:37.818089 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d8643-e42b-48a4-8fe1-99e313bf461c admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:37.941708 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:37.943983 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 811/1627] 10.222.0.78 () {58 vars in 1059 bytes} [Wed Jul 20 15:31:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:37.955373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671d63b-c84c-4694-ae05-95fde7c61194 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:37.998293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671d63b-c84c-4694-ae05-95fde7c61194 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:38.007526 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d8643-e42b-48a4-8fe1-99e313bf461c admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:38.029810 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717f7f8-cbf0-40fa-9425-ee3810a05919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:38.030874 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717f7f8-cbf0-40fa-9425-ee3810a05919 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:38.031323 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717f7f8-cbf0-40fa-9425-ee3810a05919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:38.031801 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717f7f8-cbf0-40fa-9425-ee3810a05919 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:38.032991 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717f7f8-cbf0-40fa-9425-ee3810a05919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:38.046782 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717f7f8-cbf0-40fa-9425-ee3810a05919 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:38.162741 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bec332-70c5-4f6d-9bf6-431d52571e48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:38.163646 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bec332-70c5-4f6d-9bf6-431d52571e48 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:38.164086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bec332-70c5-4f6d-9bf6-431d52571e48 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:38.165048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bec332-70c5-4f6d-9bf6-431d52571e48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:38.220227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:38.223998 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 817/1628] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:31:37 2022] POST /identity/v3/users => generated 293 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:31:38.225082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717f7f8-cbf0-40fa-9425-ee3810a05919 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:38.237419 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717f7f8-cbf0-40fa-9425-ee3810a05919 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:38.245083 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50905972-3984-4a23-8b37-9469c576c080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:38.249161 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50905972-3984-4a23-8b37-9469c576c080 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:38.249161 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50905972-3984-4a23-8b37-9469c576c080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:38.249161 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50905972-3984-4a23-8b37-9469c576c080 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:38.252426 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50905972-3984-4a23-8b37-9469c576c080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:38.258788 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50905972-3984-4a23-8b37-9469c576c080 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:38.283744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bec332-70c5-4f6d-9bf6-431d52571e48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:38.286154 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bec332-70c5-4f6d-9bf6-431d52571e48 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:38.294406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 818/1629] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:38.295691 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50905972-3984-4a23-8b37-9469c576c080 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:38.312272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 812/1630] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:38.337569 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50905972-3984-4a23-8b37-9469c576c080 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:38.353779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cbcce-88e5-46af-ad79-cc5ac386a313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:38.355287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cbcce-88e5-46af-ad79-cc5ac386a313 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:38.355572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cbcce-88e5-46af-ad79-cc5ac386a313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:38.355853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cbcce-88e5-46af-ad79-cc5ac386a313 admin admin] PATH_INFO: `/v3/projects/42b7a5e04a544e3a9836ce29c4bed110/users/120dee5b704c424a836c0c9dfbab4368/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:38.406922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71cbcce-88e5-46af-ad79-cc5ac386a313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b7a5e04a544e3a9836ce29c4bed110, user_id=120dee5b704c424a836c0c9dfbab4368, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:38.410151 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71cbcce-88e5-46af-ad79-cc5ac386a313 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:38.500184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 819/1631] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:38 2022] PUT /identity/v3/projects/42b7a5e04a544e3a9836ce29c4bed110/users/120dee5b704c424a836c0c9dfbab4368/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:38.506314 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bec332-70c5-4f6d-9bf6-431d52571e48 service nova] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:38.535805 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bec332-70c5-4f6d-9bf6-431d52571e48 service nova] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:38.667146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78b568-b9db-48e5-bd77-724f98041149 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:38.667437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78b568-b9db-48e5-bd77-724f98041149 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:38.667568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da19e2-d5c9-4d8a-a122-0ca33805ce0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:38.667718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78b568-b9db-48e5-bd77-724f98041149 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:38.669505 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 820/1632] 10.222.0.78 () {54 vars in 912 bytes} [Wed Jul 20 15:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:31:38.671619 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da19e2-d5c9-4d8a-a122-0ca33805ce0c admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:38.672305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da19e2-d5c9-4d8a-a122-0ca33805ce0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:38.673317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da19e2-d5c9-4d8a-a122-0ca33805ce0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:38.674606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da19e2-d5c9-4d8a-a122-0ca33805ce0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:38.675193 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787480e-ee51-48d6-8db8-47a635bac15f None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:38.675487 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787480e-ee51-48d6-8db8-47a635bac15f None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:38.675768 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787480e-ee51-48d6-8db8-47a635bac15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:38.683118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da19e2-d5c9-4d8a-a122-0ca33805ce0c admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:38.701729 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-4787480e-ee51-48d6-8db8-47a635bac15f None None] MFA Rules not processed for user `a4a24d5a375945779fa626fae25ac237`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:38.703626 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-4787480e-ee51-48d6-8db8-47a635bac15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:38.719378 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 813/1633] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:38.720739 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da19e2-d5c9-4d8a-a122-0ca33805ce0c admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:38.738595 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da19e2-d5c9-4d8a-a122-0ca33805ce0c admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:38.747161 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7b0ea-ddc9-4c9d-99c5-a6eaeda379fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:38.748597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7b0ea-ddc9-4c9d-99c5-a6eaeda379fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:38.748708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7b0ea-ddc9-4c9d-99c5-a6eaeda379fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:38.749118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7b0ea-ddc9-4c9d-99c5-a6eaeda379fe admin admin] PATH_INFO: `/v3/projects/42b7a5e04a544e3a9836ce29c4bed110/users/120dee5b704c424a836c0c9dfbab4368/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:38.751733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7b0ea-ddc9-4c9d-99c5-a6eaeda379fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b7a5e04a544e3a9836ce29c4bed110, user_id=120dee5b704c424a836c0c9dfbab4368, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:38.755167 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7b0ea-ddc9-4c9d-99c5-a6eaeda379fe admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:38.783935 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 814/1634] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:38 2022] PUT /identity/v3/projects/42b7a5e04a544e3a9836ce29c4bed110/users/120dee5b704c424a836c0c9dfbab4368/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:31:38.795155 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a53d4-97c7-4fea-b14e-7bc11573c3a6 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:38.795501 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a53d4-97c7-4fea-b14e-7bc11573c3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:38.798194 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a53d4-97c7-4fea-b14e-7bc11573c3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:38.873367 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-c60a53d4-97c7-4fea-b14e-7bc11573c3a6 None None] MFA Rules not processed for user `120dee5b704c424a836c0c9dfbab4368`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:38.874890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-c60a53d4-97c7-4fea-b14e-7bc11573c3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:38.883163 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 821/1635] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Jul 20 15:31:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:31:38.981230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787480e-ee51-48d6-8db8-47a635bac15f None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:39.083715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787480e-ee51-48d6-8db8-47a635bac15f None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:39.152821 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 815/1636] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:31:39.156049 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40321266-e3b3-4c83-974d-acab9a21b064 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:39.156829 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40321266-e3b3-4c83-974d-acab9a21b064 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:39.157226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40321266-e3b3-4c83-974d-acab9a21b064 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:39.157509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40321266-e3b3-4c83-974d-acab9a21b064 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:39.163814 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:39.248826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40321266-e3b3-4c83-974d-acab9a21b064 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:39.250527 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40321266-e3b3-4c83-974d-acab9a21b064 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:39.264995 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 822/1637] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:39.520829 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a53d4-97c7-4fea-b14e-7bc11573c3a6 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:39.534825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a53d4-97c7-4fea-b14e-7bc11573c3a6 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:39.542211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d8043-7cf1-4e19-84c5-af2728e9abed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:39.543786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d8043-7cf1-4e19-84c5-af2728e9abed service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:39.544882 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d8043-7cf1-4e19-84c5-af2728e9abed service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:39.545282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d8043-7cf1-4e19-84c5-af2728e9abed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:39.557481 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d8043-7cf1-4e19-84c5-af2728e9abed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:39.560500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d8043-7cf1-4e19-84c5-af2728e9abed service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:39.623107 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 816/1638] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:40.155182 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40321266-e3b3-4c83-974d-acab9a21b064 service cinder] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:40.273614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40321266-e3b3-4c83-974d-acab9a21b064 service cinder] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:40.280508 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170e70c-e84e-4904-9dc2-961922514fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:40.284247 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170e70c-e84e-4904-9dc2-961922514fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:40.284774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170e70c-e84e-4904-9dc2-961922514fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:40.286116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170e70c-e84e-4904-9dc2-961922514fca admin admin] PATH_INFO: `/v3/users/f0b4d325f04847158bffc4c8d8fdfe48` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:40.288964 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170e70c-e84e-4904-9dc2-961922514fca admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0b4d325f04847158bffc4c8d8fdfe48)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:40.291013 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: WARNING py.warnings [None req-7170e70c-e84e-4904-9dc2-961922514fca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:31:40.291013 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: warnings.warn(msg) Jul 20 15:31:40.291013 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: Jul 20 15:31:40.293153 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170e70c-e84e-4904-9dc2-961922514fca admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:40.362066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d8043-7cf1-4e19-84c5-af2728e9abed service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:40.385872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d8043-7cf1-4e19-84c5-af2728e9abed service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:40.398124 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a711b-2316-4026-871f-e16d4c8b7da5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:40.399060 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a711b-2316-4026-871f-e16d4c8b7da5 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:40.399336 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a711b-2316-4026-871f-e16d4c8b7da5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:40.399989 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a711b-2316-4026-871f-e16d4c8b7da5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:40.429423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:40.506127 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a711b-2316-4026-871f-e16d4c8b7da5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:40.508059 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a711b-2316-4026-871f-e16d4c8b7da5 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:40.518995 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 817/1639] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:31:40.837781 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7170e70c-e84e-4904-9dc2-961922514fca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0b4d325f04847158bffc4c8d8fdfe48'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:40.937687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7170e70c-e84e-4904-9dc2-961922514fca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0b4d325f04847158bffc4c8d8fdfe48'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:40.996193 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7170e70c-e84e-4904-9dc2-961922514fca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0b4d325f04847158bffc4c8d8fdfe48'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:40.998502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7170e70c-e84e-4904-9dc2-961922514fca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0b4d325f04847158bffc4c8d8fdfe48'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:41.045139 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7170e70c-e84e-4904-9dc2-961922514fca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0b4d325f04847158bffc4c8d8fdfe48'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:41.070625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 823/1640] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:31:40 2022] DELETE /identity/v3/users/f0b4d325f04847158bffc4c8d8fdfe48 => generated 0 bytes in 911 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:41.070933 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937ee86-cb64-4f46-8236-e6e9f5802cae None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:41.071416 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937ee86-cb64-4f46-8236-e6e9f5802cae None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:41.071517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937ee86-cb64-4f46-8236-e6e9f5802cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:41.143779 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-b937ee86-cb64-4f46-8236-e6e9f5802cae None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:41.148597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b937ee86-cb64-4f46-8236-e6e9f5802cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:41.467475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 818/1641] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:41.555630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170e70c-e84e-4904-9dc2-961922514fca admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:41.557822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-7170e70c-e84e-4904-9dc2-961922514fca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:41.586287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:41.715381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170e70c-e84e-4904-9dc2-961922514fca admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:41.721480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338c18d-d25f-4f41-a200-5fe9ae32b824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:41.722443 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338c18d-d25f-4f41-a200-5fe9ae32b824 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:41.722914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338c18d-d25f-4f41-a200-5fe9ae32b824 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:41.723419 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338c18d-d25f-4f41-a200-5fe9ae32b824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:41.740812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338c18d-d25f-4f41-a200-5fe9ae32b824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:41.745024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338c18d-d25f-4f41-a200-5fe9ae32b824 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:41.761272 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 824/1642] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:41.805005 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b193286-56ad-4c11-bfcb-cae2bf97555a None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:41.805706 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b193286-56ad-4c11-bfcb-cae2bf97555a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:41.806328 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b193286-56ad-4c11-bfcb-cae2bf97555a None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:41.811154 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 819/1643] 10.222.0.78 () {54 vars in 962 bytes} [Wed Jul 20 15:31:41 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:31:41.837753 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e712e9-e8e7-4157-a648-a6308ffc85b2 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:41.838067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e712e9-e8e7-4157-a648-a6308ffc85b2 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:41.838385 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e712e9-e8e7-4157-a648-a6308ffc85b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:41.871696 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-c7e712e9-e8e7-4157-a648-a6308ffc85b2 None None] MFA Rules not processed for user `a4a24d5a375945779fa626fae25ac237`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:41.873520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c7e712e9-e8e7-4157-a648-a6308ffc85b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:42.149290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 825/1644] 10.222.0.78 () {58 vars in 1059 bytes} [Wed Jul 20 15:31:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:42.161891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f499d8e-69a9-445c-b27f-153726c8a298 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:42.162775 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f499d8e-69a9-445c-b27f-153726c8a298 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:42.162775 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f499d8e-69a9-445c-b27f-153726c8a298 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:42.164196 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 820/1645] 10.222.0.78 () {54 vars in 912 bytes} [Wed Jul 20 15:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:31:42.185899 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe473ec-43cd-4730-b3ed-d3c5d9a90b87 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:42.187202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe473ec-43cd-4730-b3ed-d3c5d9a90b87 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:42.188033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe473ec-43cd-4730-b3ed-d3c5d9a90b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:42.216107 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-2fe473ec-43cd-4730-b3ed-d3c5d9a90b87 None None] MFA Rules not processed for user `a4a24d5a375945779fa626fae25ac237`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:42.218820 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2fe473ec-43cd-4730-b3ed-d3c5d9a90b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:42.239956 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 826/1646] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Jul 20 15:31:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:42.266326 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f499d8e-69a9-445c-b27f-153726c8a298 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:42.269316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-1f499d8e-69a9-445c-b27f-153726c8a298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:42.442362 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f499d8e-69a9-445c-b27f-153726c8a298 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:42.448653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2eb99-c3dc-4db4-904f-1613ee4e3d13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:42.449540 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2eb99-c3dc-4db4-904f-1613ee4e3d13 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:42.449827 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2eb99-c3dc-4db4-904f-1613ee4e3d13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:42.450108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2eb99-c3dc-4db4-904f-1613ee4e3d13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:42.484424 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2eb99-c3dc-4db4-904f-1613ee4e3d13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:42.487251 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2eb99-c3dc-4db4-904f-1613ee4e3d13 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:42.516728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 821/1647] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:43.199392 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe473ec-43cd-4730-b3ed-d3c5d9a90b87 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:43.200924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2fe473ec-43cd-4730-b3ed-d3c5d9a90b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:43.232267 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe473ec-43cd-4730-b3ed-d3c5d9a90b87 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:43.250038 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4328f8f-19be-4daf-a4fb-ab07a5494230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:43.251624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4328f8f-19be-4daf-a4fb-ab07a5494230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:43.256432 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4328f8f-19be-4daf-a4fb-ab07a5494230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:43.256432 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4328f8f-19be-4daf-a4fb-ab07a5494230 admin admin] PATH_INFO: `/v3/projects/f10025ba797e4e21a8ebf27435eea153` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:43.257409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4328f8f-19be-4daf-a4fb-ab07a5494230 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f10025ba797e4e21a8ebf27435eea153)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:43.265909 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4328f8f-19be-4daf-a4fb-ab07a5494230 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:43.524656 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b4328f8f-19be-4daf-a4fb-ab07a5494230 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f10025ba797e4e21a8ebf27435eea153', 'request_id': 'req-b4328f8f-19be-4daf-a4fb-ab07a5494230'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:43.650200 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b4328f8f-19be-4daf-a4fb-ab07a5494230 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f10025ba797e4e21a8ebf27435eea153', 'request_id': 'req-b4328f8f-19be-4daf-a4fb-ab07a5494230'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:43.663059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b4328f8f-19be-4daf-a4fb-ab07a5494230 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f10025ba797e4e21a8ebf27435eea153', 'request_id': 'req-b4328f8f-19be-4daf-a4fb-ab07a5494230'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:43.694423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b4328f8f-19be-4daf-a4fb-ab07a5494230 admin admin] The token cache is being invalidate because project f10025ba797e4e21a8ebf27435eea153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:31:43.696445 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b4328f8f-19be-4daf-a4fb-ab07a5494230 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:43.698965 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b4328f8f-19be-4daf-a4fb-ab07a5494230 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:43.713793 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 827/1648] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:31:43 2022] DELETE /identity/v3/projects/f10025ba797e4e21a8ebf27435eea153 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:43.716118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2eb99-c3dc-4db4-904f-1613ee4e3d13 service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:43.738648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2eb99-c3dc-4db4-904f-1613ee4e3d13 service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:43.899014 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ff0df-d08a-4fb6-9182-63b80035d3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:43.899927 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ff0df-d08a-4fb6-9182-63b80035d3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:43.900216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ff0df-d08a-4fb6-9182-63b80035d3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:43.900525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ff0df-d08a-4fb6-9182-63b80035d3fe admin admin] PATH_INFO: `/v3/users/69c0f9504d6c434a99cc2671c483f689` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:43.902139 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ff0df-d08a-4fb6-9182-63b80035d3fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=69c0f9504d6c434a99cc2671c483f689)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:43.909032 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ff0df-d08a-4fb6-9182-63b80035d3fe admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:43.910991 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:44.234668 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-cc0ff0df-d08a-4fb6-9182-63b80035d3fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69c0f9504d6c434a99cc2671c483f689'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:44.330425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-cc0ff0df-d08a-4fb6-9182-63b80035d3fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69c0f9504d6c434a99cc2671c483f689'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:44.334495 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-cc0ff0df-d08a-4fb6-9182-63b80035d3fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69c0f9504d6c434a99cc2671c483f689'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:44.407257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-cc0ff0df-d08a-4fb6-9182-63b80035d3fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69c0f9504d6c434a99cc2671c483f689'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:44.440001 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-cc0ff0df-d08a-4fb6-9182-63b80035d3fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69c0f9504d6c434a99cc2671c483f689'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:44.477925 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 822/1649] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:31:43 2022] DELETE /identity/v3/users/69c0f9504d6c434a99cc2671c483f689 => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:45.498634 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4328f8f-19be-4daf-a4fb-ab07a5494230 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:45.500370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b4328f8f-19be-4daf-a4fb-ab07a5494230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:45.522515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4328f8f-19be-4daf-a4fb-ab07a5494230 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:45.591025 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:45.685169 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79813ad7-da24-4311-971a-d53b3b0c9682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:45.686243 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79813ad7-da24-4311-971a-d53b3b0c9682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:45.686624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79813ad7-da24-4311-971a-d53b3b0c9682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:45.686995 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79813ad7-da24-4311-971a-d53b3b0c9682 admin admin] PATH_INFO: `/v3/projects/6afe2cf697fd469d97865059a2700ed6` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:45.689551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79813ad7-da24-4311-971a-d53b3b0c9682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6afe2cf697fd469d97865059a2700ed6)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:45.691702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79813ad7-da24-4311-971a-d53b3b0c9682 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:46.004782 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-79813ad7-da24-4311-971a-d53b3b0c9682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6afe2cf697fd469d97865059a2700ed6', 'request_id': 'req-79813ad7-da24-4311-971a-d53b3b0c9682'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:46.098603 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-79813ad7-da24-4311-971a-d53b3b0c9682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6afe2cf697fd469d97865059a2700ed6', 'request_id': 'req-79813ad7-da24-4311-971a-d53b3b0c9682'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:46.133247 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-79813ad7-da24-4311-971a-d53b3b0c9682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6afe2cf697fd469d97865059a2700ed6', 'request_id': 'req-79813ad7-da24-4311-971a-d53b3b0c9682'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:46.156725 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-79813ad7-da24-4311-971a-d53b3b0c9682 admin admin] The token cache is being invalidate because project 6afe2cf697fd469d97865059a2700ed6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:31:46.157390 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-79813ad7-da24-4311-971a-d53b3b0c9682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:46.159168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-79813ad7-da24-4311-971a-d53b3b0c9682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:46.171292 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 828/1650] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:31:45 2022] DELETE /identity/v3/projects/6afe2cf697fd469d97865059a2700ed6 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:46.171858 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552000b-16e9-4c61-8144-8cef33c95de0 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:46.180801 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552000b-16e9-4c61-8144-8cef33c95de0 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:46.182534 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552000b-16e9-4c61-8144-8cef33c95de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:46.219301 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-4552000b-16e9-4c61-8144-8cef33c95de0 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:46.220604 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4552000b-16e9-4c61-8144-8cef33c95de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:46.355660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a717ac6-7236-43f2-af3b-8f59f2d66ccd None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:46.356467 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a717ac6-7236-43f2-af3b-8f59f2d66ccd None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:46.358455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a717ac6-7236-43f2-af3b-8f59f2d66ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:46.390537 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 823/1651] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:46.419005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-3a717ac6-7236-43f2-af3b-8f59f2d66ccd None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:46.419005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-3a717ac6-7236-43f2-af3b-8f59f2d66ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:46.443369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 829/1652] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:46.444479 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552000b-16e9-4c61-8144-8cef33c95de0 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:46.467348 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552000b-16e9-4c61-8144-8cef33c95de0 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:46.478249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3abef-651b-4fcf-91f1-7bd2096cc49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:46.480048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3abef-651b-4fcf-91f1-7bd2096cc49e admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:46.480542 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3abef-651b-4fcf-91f1-7bd2096cc49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:46.481023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3abef-651b-4fcf-91f1-7bd2096cc49e admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:46.482077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3abef-651b-4fcf-91f1-7bd2096cc49e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:31:46.482649 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3abef-651b-4fcf-91f1-7bd2096cc49e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:46.484843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3abef-651b-4fcf-91f1-7bd2096cc49e admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:46.512199 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 824/1653] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:31:46.518549 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a717ac6-7236-43f2-af3b-8f59f2d66ccd None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:46.540192 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a717ac6-7236-43f2-af3b-8f59f2d66ccd None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:46.555115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07429f-8365-4f65-8259-e74aa80cc4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:46.555115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07429f-8365-4f65-8259-e74aa80cc4ca admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:46.555115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07429f-8365-4f65-8259-e74aa80cc4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:46.555115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07429f-8365-4f65-8259-e74aa80cc4ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:46.555688 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07429f-8365-4f65-8259-e74aa80cc4ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:46.555688 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07429f-8365-4f65-8259-e74aa80cc4ca admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:46.688064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 830/1654] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:31:46 2022] POST /identity/v3/projects => generated 363 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:46.693763 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3abef-651b-4fcf-91f1-7bd2096cc49e admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:46.714884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3abef-651b-4fcf-91f1-7bd2096cc49e admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:46.845662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab32208-a172-40f2-b7a2-50747699dba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:46.846734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab32208-a172-40f2-b7a2-50747699dba3 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:46.847011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab32208-a172-40f2-b7a2-50747699dba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:46.847180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab32208-a172-40f2-b7a2-50747699dba3 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:46.848141 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab32208-a172-40f2-b7a2-50747699dba3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:46.850369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab32208-a172-40f2-b7a2-50747699dba3 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:46.881000 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:47.017946 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 825/1655] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:31:46 2022] POST /identity/v3/users => generated 282 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:47.022401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07429f-8365-4f65-8259-e74aa80cc4ca admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:47.044753 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07429f-8365-4f65-8259-e74aa80cc4ca admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:47.087017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba7120-be46-4e11-80f4-5977e18a0f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:47.114832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba7120-be46-4e11-80f4-5977e18a0f56 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:47.118803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba7120-be46-4e11-80f4-5977e18a0f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:47.124856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba7120-be46-4e11-80f4-5977e18a0f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:47.125460 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba7120-be46-4e11-80f4-5977e18a0f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:47.127925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba7120-be46-4e11-80f4-5977e18a0f56 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:47.156194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 831/1656] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:47.159326 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab32208-a172-40f2-b7a2-50747699dba3 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:47.186815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab32208-a172-40f2-b7a2-50747699dba3 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:47.200553 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782fcd8-fb1f-457c-8887-abea252f88eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:47.201589 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782fcd8-fb1f-457c-8887-abea252f88eb admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:47.201990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782fcd8-fb1f-457c-8887-abea252f88eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:47.202637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782fcd8-fb1f-457c-8887-abea252f88eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:47.203536 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782fcd8-fb1f-457c-8887-abea252f88eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:47.209289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782fcd8-fb1f-457c-8887-abea252f88eb admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:47.259113 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 826/1657] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:47.264743 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba7120-be46-4e11-80f4-5977e18a0f56 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:47.283253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba7120-be46-4e11-80f4-5977e18a0f56 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:47.299067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17721080-49f0-4b46-a18f-f932d07697b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:47.300030 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17721080-49f0-4b46-a18f-f932d07697b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:47.300320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17721080-49f0-4b46-a18f-f932d07697b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:47.300603 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17721080-49f0-4b46-a18f-f932d07697b7 admin admin] PATH_INFO: `/v3/projects/bd335124dde5462b970264a96448f422/users/6227bd78716a4a6684fe0672ecff35f5/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:47.335101 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17721080-49f0-4b46-a18f-f932d07697b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd335124dde5462b970264a96448f422, user_id=6227bd78716a4a6684fe0672ecff35f5, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:47.339818 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17721080-49f0-4b46-a18f-f932d07697b7 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:47.472992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 832/1658] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:47 2022] PUT /identity/v3/projects/bd335124dde5462b970264a96448f422/users/6227bd78716a4a6684fe0672ecff35f5/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:47.484934 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174765f-067c-463c-8447-d77d47790937 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:47.486468 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174765f-067c-463c-8447-d77d47790937 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:47.487024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174765f-067c-463c-8447-d77d47790937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:47.569132 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-0174765f-067c-463c-8447-d77d47790937 None None] MFA Rules not processed for user `6227bd78716a4a6684fe0672ecff35f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:47.571577 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0174765f-067c-463c-8447-d77d47790937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:47.698996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 827/1659] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:47 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:47.703536 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17721080-49f0-4b46-a18f-f932d07697b7 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:47.916828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17721080-49f0-4b46-a18f-f932d07697b7 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:47.929801 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19d879-8a10-421a-bb5e-f8c763e803e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:47.931430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19d879-8a10-421a-bb5e-f8c763e803e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:47.932407 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19d879-8a10-421a-bb5e-f8c763e803e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:47.932683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19d879-8a10-421a-bb5e-f8c763e803e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:47.933891 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19d879-8a10-421a-bb5e-f8c763e803e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:47.936754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19d879-8a10-421a-bb5e-f8c763e803e0 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:48.132404 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 833/1660] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:31:47 2022] POST /identity/v3/projects => generated 365 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:31:48.138076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174765f-067c-463c-8447-d77d47790937 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:48.153748 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174765f-067c-463c-8447-d77d47790937 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:48.200912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:48.373334 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db450480-cc12-4c63-b70b-c8ae0b93d776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:48.376116 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db450480-cc12-4c63-b70b-c8ae0b93d776 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:48.376407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db450480-cc12-4c63-b70b-c8ae0b93d776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:48.376689 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db450480-cc12-4c63-b70b-c8ae0b93d776 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:48.380793 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db450480-cc12-4c63-b70b-c8ae0b93d776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:48.391494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db450480-cc12-4c63-b70b-c8ae0b93d776 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:48.555981 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 828/1661] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:31:48 2022] POST /identity/v3/users => generated 289 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:31:48.560800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19d879-8a10-421a-bb5e-f8c763e803e0 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:48.574506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19d879-8a10-421a-bb5e-f8c763e803e0 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:48.581414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245e003-5e01-48f7-8417-c7d8bbf5edd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:48.584929 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245e003-5e01-48f7-8417-c7d8bbf5edd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:48.585223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245e003-5e01-48f7-8417-c7d8bbf5edd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:48.585507 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245e003-5e01-48f7-8417-c7d8bbf5edd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:48.586544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245e003-5e01-48f7-8417-c7d8bbf5edd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:48.588773 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245e003-5e01-48f7-8417-c7d8bbf5edd1 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:48.620720 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 834/1662] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:48.625235 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db450480-cc12-4c63-b70b-c8ae0b93d776 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:48.646134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db450480-cc12-4c63-b70b-c8ae0b93d776 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:48.662480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee140d-bd79-475b-8527-02379e89ab63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:48.663413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee140d-bd79-475b-8527-02379e89ab63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:48.663713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee140d-bd79-475b-8527-02379e89ab63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:48.663997 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee140d-bd79-475b-8527-02379e89ab63 admin admin] PATH_INFO: `/v3/projects/ea5333d99a3d4bf3acf91f096268f4b9/users/0052a6855c594ed7a24450b41d0a5bc9/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:48.708814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee140d-bd79-475b-8527-02379e89ab63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea5333d99a3d4bf3acf91f096268f4b9, user_id=0052a6855c594ed7a24450b41d0a5bc9, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:48.725163 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee140d-bd79-475b-8527-02379e89ab63 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:48.889502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 829/1663] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:48 2022] PUT /identity/v3/projects/ea5333d99a3d4bf3acf91f096268f4b9/users/0052a6855c594ed7a24450b41d0a5bc9/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:48.892920 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245e003-5e01-48f7-8417-c7d8bbf5edd1 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:48.930114 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245e003-5e01-48f7-8417-c7d8bbf5edd1 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:49.124616 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1c861-43e6-44fd-a47f-d20239ea0527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:49.125367 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1c861-43e6-44fd-a47f-d20239ea0527 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:49.130625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1c861-43e6-44fd-a47f-d20239ea0527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:49.130625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1c861-43e6-44fd-a47f-d20239ea0527 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:49.131841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1c861-43e6-44fd-a47f-d20239ea0527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:49.133537 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1c861-43e6-44fd-a47f-d20239ea0527 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:49.154711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 835/1664] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:31:49.155998 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee140d-bd79-475b-8527-02379e89ab63 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:49.166754 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee140d-bd79-475b-8527-02379e89ab63 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:49.195879 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8cf1c-c1ec-4f03-9d77-64b5f30877d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:49.197959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8cf1c-c1ec-4f03-9d77-64b5f30877d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:49.199387 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8cf1c-c1ec-4f03-9d77-64b5f30877d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:49.199887 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8cf1c-c1ec-4f03-9d77-64b5f30877d8 admin admin] PATH_INFO: `/v3/projects/ea5333d99a3d4bf3acf91f096268f4b9/users/0052a6855c594ed7a24450b41d0a5bc9/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:49.208782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8cf1c-c1ec-4f03-9d77-64b5f30877d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea5333d99a3d4bf3acf91f096268f4b9, user_id=0052a6855c594ed7a24450b41d0a5bc9, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:49.212113 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8cf1c-c1ec-4f03-9d77-64b5f30877d8 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:49.241286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 830/1665] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:31:49 2022] PUT /identity/v3/projects/ea5333d99a3d4bf3acf91f096268f4b9/users/0052a6855c594ed7a24450b41d0a5bc9/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:49.257097 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d656c2a-cbe0-40e9-bfbf-6e3dbd43880d None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:49.257097 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d656c2a-cbe0-40e9-bfbf-6e3dbd43880d None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:49.257097 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d656c2a-cbe0-40e9-bfbf-6e3dbd43880d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:49.316456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-7d656c2a-cbe0-40e9-bfbf-6e3dbd43880d None None] MFA Rules not processed for user `0052a6855c594ed7a24450b41d0a5bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:49.317819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-7d656c2a-cbe0-40e9-bfbf-6e3dbd43880d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:49.490540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 836/1666] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:49 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:31:49.554480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8cf1c-c1ec-4f03-9d77-64b5f30877d8 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:49.556910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-16c8cf1c-c1ec-4f03-9d77-64b5f30877d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:49.598039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:49.630033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8cf1c-c1ec-4f03-9d77-64b5f30877d8 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:49.634377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fd121-260e-4338-bd6e-4c90771ebf54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:49.635942 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fd121-260e-4338-bd6e-4c90771ebf54 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:49.636414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fd121-260e-4338-bd6e-4c90771ebf54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:49.636847 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fd121-260e-4338-bd6e-4c90771ebf54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:49.648185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67fd121-260e-4338-bd6e-4c90771ebf54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:49.649652 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67fd121-260e-4338-bd6e-4c90771ebf54 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:49.663086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 831/1667] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:49 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:49.988760 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d656c2a-cbe0-40e9-bfbf-6e3dbd43880d None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:49.991206 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-7d656c2a-cbe0-40e9-bfbf-6e3dbd43880d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:50.030562 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d656c2a-cbe0-40e9-bfbf-6e3dbd43880d None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:50.035522 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa26c0e-23b7-4e36-be6e-35aaf618ce9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:50.036997 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa26c0e-23b7-4e36-be6e-35aaf618ce9e service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:50.037484 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa26c0e-23b7-4e36-be6e-35aaf618ce9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:50.037963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa26c0e-23b7-4e36-be6e-35aaf618ce9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:50.140475 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:50.171915 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa26c0e-23b7-4e36-be6e-35aaf618ce9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:50.173491 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa26c0e-23b7-4e36-be6e-35aaf618ce9e service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:50.214038 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 837/1668] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:49 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:51.200063 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b55f7-1baa-4837-883d-9ae5ce9ea238 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:51.204991 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b55f7-1baa-4837-883d-9ae5ce9ea238 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:51.204991 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b55f7-1baa-4837-883d-9ae5ce9ea238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:51.271065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-860b55f7-1baa-4837-883d-9ae5ce9ea238 None None] MFA Rules not processed for user `8e75c180bae74be0b2ccad079f92d32b`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:51.272720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-860b55f7-1baa-4837-883d-9ae5ce9ea238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:51.287003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f57cff-3dfe-4a9b-be73-4cfa16baf1af None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:51.287003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f57cff-3dfe-4a9b-be73-4cfa16baf1af None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:51.287003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f57cff-3dfe-4a9b-be73-4cfa16baf1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:51.352224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-b2f57cff-3dfe-4a9b-be73-4cfa16baf1af None None] MFA Rules not processed for user `ebbcfe6341a3486d93e12c09f83954dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:51.354087 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b2f57cff-3dfe-4a9b-be73-4cfa16baf1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:51.434401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 832/1669] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:51.449698 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b55f7-1baa-4837-883d-9ae5ce9ea238 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:51.457306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-860b55f7-1baa-4837-883d-9ae5ce9ea238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:51.578301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 838/1670] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:51.583786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a96786-9604-46fa-b03d-dac381b935d3 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:51.590633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a96786-9604-46fa-b03d-dac381b935d3 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:51.590633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a96786-9604-46fa-b03d-dac381b935d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:51.641476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-97a96786-9604-46fa-b03d-dac381b935d3 None None] MFA Rules not processed for user `ebbcfe6341a3486d93e12c09f83954dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:51.642826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-97a96786-9604-46fa-b03d-dac381b935d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:51.665892 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 839/1671] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:31:51.676420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a96786-9604-46fa-b03d-dac381b935d3 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:51.678464 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-97a96786-9604-46fa-b03d-dac381b935d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:51.679560 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b55f7-1baa-4837-883d-9ae5ce9ea238 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:51.696289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da71e0b-8a5c-45e0-9073-ca0d0cebfef3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:51.704851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da71e0b-8a5c-45e0-9073-ca0d0cebfef3 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:51.704851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da71e0b-8a5c-45e0-9073-ca0d0cebfef3 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:51.704851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da71e0b-8a5c-45e0-9073-ca0d0cebfef3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:51.715967 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a96786-9604-46fa-b03d-dac381b935d3 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:51.720342 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da71e0b-8a5c-45e0-9073-ca0d0cebfef3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:51.725158 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da71e0b-8a5c-45e0-9073-ca0d0cebfef3 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:51.740053 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb99821-abb7-442e-bac6-0d56c5667c89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:51.741048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb99821-abb7-442e-bac6-0d56c5667c89 service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:51.741489 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb99821-abb7-442e-bac6-0d56c5667c89 service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:51.743780 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb99821-abb7-442e-bac6-0d56c5667c89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:51.756610 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 833/1672] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:51.765451 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb99821-abb7-442e-bac6-0d56c5667c89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:51.769101 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb99821-abb7-442e-bac6-0d56c5667c89 service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:51.795601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 840/1673] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:31:53.167784 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbdcef-9128-42a6-aa5e-5284a0194862 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:53.168286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbdcef-9128-42a6-aa5e-5284a0194862 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:53.168559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbdcef-9128-42a6-aa5e-5284a0194862 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:53.171042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 834/1674] 10.222.0.78 () {54 vars in 912 bytes} [Wed Jul 20 15:31:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:31:53.178570 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ca87d-c16f-4b56-a483-e25e1a64dff7 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:53.179054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ca87d-c16f-4b56-a483-e25e1a64dff7 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:53.180137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ca87d-c16f-4b56-a483-e25e1a64dff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:53.272891 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-f79ca87d-c16f-4b56-a483-e25e1a64dff7 None None] MFA Rules not processed for user `b57530f18d9c4d328f8014db736d8de9`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:53.274360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f79ca87d-c16f-4b56-a483-e25e1a64dff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:53.457278 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 841/1675] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Jul 20 15:31:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:53.475096 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbdcef-9128-42a6-aa5e-5284a0194862 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:53.478149 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-6acbdcef-9128-42a6-aa5e-5284a0194862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:53.503248 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbdcef-9128-42a6-aa5e-5284a0194862 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:53.507782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ac7ce-d127-48b9-864c-a0580a5097d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:53.508843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ac7ce-d127-48b9-864c-a0580a5097d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:53.511166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ac7ce-d127-48b9-864c-a0580a5097d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:53.513094 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ac7ce-d127-48b9-864c-a0580a5097d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:53.567443 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320ac7ce-d127-48b9-864c-a0580a5097d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:53.575649 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320ac7ce-d127-48b9-864c-a0580a5097d7 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:53.628838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 835/1676] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:54.071003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ca87d-c16f-4b56-a483-e25e1a64dff7 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:54.072502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f79ca87d-c16f-4b56-a483-e25e1a64dff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:54.235317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:54.410996 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ca87d-c16f-4b56-a483-e25e1a64dff7 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:54.410996 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda83b14-5187-47ce-871a-c248d4ad7639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:54.410996 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda83b14-5187-47ce-871a-c248d4ad7639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:54.410996 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda83b14-5187-47ce-871a-c248d4ad7639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:54.411513 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda83b14-5187-47ce-871a-c248d4ad7639 admin admin] PATH_INFO: `/v3/users/ca0d143cb0154a0d854bdb9509216395` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:54.411513 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda83b14-5187-47ce-871a-c248d4ad7639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca0d143cb0154a0d854bdb9509216395)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:54.411513 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda83b14-5187-47ce-871a-c248d4ad7639 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:54.864841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-dda83b14-5187-47ce-871a-c248d4ad7639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca0d143cb0154a0d854bdb9509216395'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:54.954104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-dda83b14-5187-47ce-871a-c248d4ad7639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca0d143cb0154a0d854bdb9509216395'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:54.970060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-dda83b14-5187-47ce-871a-c248d4ad7639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca0d143cb0154a0d854bdb9509216395'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:54.971924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-dda83b14-5187-47ce-871a-c248d4ad7639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca0d143cb0154a0d854bdb9509216395'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:55.056448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-dda83b14-5187-47ce-871a-c248d4ad7639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca0d143cb0154a0d854bdb9509216395'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:55.091322 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 842/1677] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:31:54 2022] DELETE /identity/v3/users/ca0d143cb0154a0d854bdb9509216395 => generated 0 bytes in 1021 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:55.097789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e45d6-523b-470c-a40e-ea2deee7e9ca None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:55.104025 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e45d6-523b-470c-a40e-ea2deee7e9ca None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:55.104025 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e45d6-523b-470c-a40e-ea2deee7e9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:55.216078 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-d95e45d6-523b-470c-a40e-ea2deee7e9ca None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:55.237886 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d95e45d6-523b-470c-a40e-ea2deee7e9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:55.313631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332aa7a-512b-467a-989b-8f1765031838 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:55.314559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332aa7a-512b-467a-989b-8f1765031838 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:55.314559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332aa7a-512b-467a-989b-8f1765031838 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:55.317237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 843/1678] 10.222.0.78 () {54 vars in 961 bytes} [Wed Jul 20 15:31:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 15:31:55.384188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cdf11-dd7d-46ab-b744-0318b9ec1bb8 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:55.386749 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cdf11-dd7d-46ab-b744-0318b9ec1bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:55.387981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cdf11-dd7d-46ab-b744-0318b9ec1bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:55.485502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:55.530890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-d50cdf11-dd7d-46ab-b744-0318b9ec1bb8 None None] MFA Rules not processed for user `2192547babde4449a46f9cb37ed64524`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:55.546736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d50cdf11-dd7d-46ab-b744-0318b9ec1bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:55.652722 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 836/1679] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 555 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:55.714789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e45d6-523b-470c-a40e-ea2deee7e9ca None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:55.716188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d95e45d6-523b-470c-a40e-ea2deee7e9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:55.922190 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 844/1680] 10.222.0.78 () {58 vars in 1058 bytes} [Wed Jul 20 15:31:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 541 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:55.932317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4e066-2d48-403b-a336-20f73c763682 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:55.938156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e45d6-523b-470c-a40e-ea2deee7e9ca None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:55.938156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2d4af-fd35-48b8-8147-c862cea29641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:55.939242 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2d4af-fd35-48b8-8147-c862cea29641 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:55.940375 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2d4af-fd35-48b8-8147-c862cea29641 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:55.941006 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2d4af-fd35-48b8-8147-c862cea29641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:55.943805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4e066-2d48-403b-a336-20f73c763682 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:55.946608 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4e066-2d48-403b-a336-20f73c763682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:55.955453 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2d4af-fd35-48b8-8147-c862cea29641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:55.957783 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2d4af-fd35-48b8-8147-c862cea29641 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:55.976924 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 837/1681] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:55.978901 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2d4af-fd35-48b8-8147-c862cea29641 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:55.994795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2d4af-fd35-48b8-8147-c862cea29641 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:56.005264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213dfa7-b5b4-40ff-bfb7-4e3c031504ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2192547babde4449a46f9cb37ed64524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:56.006174 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213dfa7-b5b4-40ff-bfb7-4e3c031504ea service placement] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:56.006645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213dfa7-b5b4-40ff-bfb7-4e3c031504ea service placement] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:56.007309 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213dfa7-b5b4-40ff-bfb7-4e3c031504ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:56.031956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-9213dfa7-b5b4-40ff-bfb7-4e3c031504ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:56.057149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-dad4e066-2d48-403b-a336-20f73c763682 None None] MFA Rules not processed for user `ebbcfe6341a3486d93e12c09f83954dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:56.062507 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-dad4e066-2d48-403b-a336-20f73c763682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:56.073349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213dfa7-b5b4-40ff-bfb7-4e3c031504ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:56.085652 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213dfa7-b5b4-40ff-bfb7-4e3c031504ea service placement] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:56.244564 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 838/1682] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:56.278405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 845/1683] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:57.305972 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e488a-78ef-438d-9235-11eebf84ce83 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:57.307538 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e488a-78ef-438d-9235-11eebf84ce83 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:57.308007 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e488a-78ef-438d-9235-11eebf84ce83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:57.376859 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-c16e488a-78ef-438d-9235-11eebf84ce83 None None] MFA Rules not processed for user `6498ef5c517c4fa4abed90e430936ded`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:57.378851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-c16e488a-78ef-438d-9235-11eebf84ce83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:57.588523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 839/1684] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:57.594589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4e066-2d48-403b-a336-20f73c763682 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:57.599861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-dad4e066-2d48-403b-a336-20f73c763682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:57.629604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:57.760394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4e066-2d48-403b-a336-20f73c763682 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:57.777842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e8dae-7d6d-43c4-ae49-a23e1b777d9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:57.782629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e8dae-7d6d-43c4-ae49-a23e1b777d9d service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:57.782629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e8dae-7d6d-43c4-ae49-a23e1b777d9d service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:57.782629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e8dae-7d6d-43c4-ae49-a23e1b777d9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:57.819382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e8dae-7d6d-43c4-ae49-a23e1b777d9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:57.821135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e8dae-7d6d-43c4-ae49-a23e1b777d9d service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:57.844808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 846/1685] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:57.880082 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b6fc8-6149-4c9f-b1da-35b6ae87ff8e None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:57.880513 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b6fc8-6149-4c9f-b1da-35b6ae87ff8e None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:57.881140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b6fc8-6149-4c9f-b1da-35b6ae87ff8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:57.955470 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e8dae-7d6d-43c4-ae49-a23e1b777d9d service nova] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:57.957990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6b5e8dae-7d6d-43c4-ae49-a23e1b777d9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:58.007499 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-1a6b6fc8-6149-4c9f-b1da-35b6ae87ff8e None None] MFA Rules not processed for user `48ce3bfba3724b07a657774662b8d6f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:58.009974 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-1a6b6fc8-6149-4c9f-b1da-35b6ae87ff8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:58.033544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e8dae-7d6d-43c4-ae49-a23e1b777d9d service nova] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:58.084379 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca4770-c1f9-4488-83b4-a2c07d1f7674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:58.086368 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca4770-c1f9-4488-83b4-a2c07d1f7674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:58.088630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca4770-c1f9-4488-83b4-a2c07d1f7674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:58.088630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca4770-c1f9-4488-83b4-a2c07d1f7674 admin admin] PATH_INFO: `/v3/users/5140c1159e6f46959cc44b809074c61a` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:58.096964 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca4770-c1f9-4488-83b4-a2c07d1f7674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5140c1159e6f46959cc44b809074c61a)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:58.118752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca4770-c1f9-4488-83b4-a2c07d1f7674 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:58.197769 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:58.304930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 840/1686] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:58.305243 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b6fc8-6149-4c9f-b1da-35b6ae87ff8e None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:58.310370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-1a6b6fc8-6149-4c9f-b1da-35b6ae87ff8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:58.396781 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b6fc8-6149-4c9f-b1da-35b6ae87ff8e None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:58.405885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14590e-b8d6-4125-89a7-250c6e71a6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:58.407265 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14590e-b8d6-4125-89a7-250c6e71a6dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:58.407693 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14590e-b8d6-4125-89a7-250c6e71a6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:58.408129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14590e-b8d6-4125-89a7-250c6e71a6dc admin admin] PATH_INFO: `/v3/users/6227bd78716a4a6684fe0672ecff35f5` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:58.412205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14590e-b8d6-4125-89a7-250c6e71a6dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6227bd78716a4a6684fe0672ecff35f5)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:58.414428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14590e-b8d6-4125-89a7-250c6e71a6dc admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:58.558110 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7dca4770-c1f9-4488-83b4-a2c07d1f7674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5140c1159e6f46959cc44b809074c61a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:58.656467 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7dca4770-c1f9-4488-83b4-a2c07d1f7674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5140c1159e6f46959cc44b809074c61a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:58.669765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7dca4770-c1f9-4488-83b4-a2c07d1f7674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5140c1159e6f46959cc44b809074c61a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:58.670824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7dca4770-c1f9-4488-83b4-a2c07d1f7674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5140c1159e6f46959cc44b809074c61a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:58.702756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7dca4770-c1f9-4488-83b4-a2c07d1f7674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5140c1159e6f46959cc44b809074c61a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:58.725756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 847/1687] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:31:57 2022] DELETE /identity/v3/users/5140c1159e6f46959cc44b809074c61a => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:31:58.729955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ff68d-91dd-445f-bbdd-4f139d60aedf None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:58.734656 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ff68d-91dd-445f-bbdd-4f139d60aedf None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:58.734656 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ff68d-91dd-445f-bbdd-4f139d60aedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:58.753051 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-1c14590e-b8d6-4125-89a7-250c6e71a6dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6227bd78716a4a6684fe0672ecff35f5'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:58.773291 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-9c6ff68d-91dd-445f-bbdd-4f139d60aedf None None] MFA Rules not processed for user `48ce3bfba3724b07a657774662b8d6f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:58.775342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9c6ff68d-91dd-445f-bbdd-4f139d60aedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:58.896104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-1c14590e-b8d6-4125-89a7-250c6e71a6dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6227bd78716a4a6684fe0672ecff35f5'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:58.897102 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-1c14590e-b8d6-4125-89a7-250c6e71a6dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6227bd78716a4a6684fe0672ecff35f5'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:58.924376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 848/1688] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:58.932563 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ff68d-91dd-445f-bbdd-4f139d60aedf None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:58.940474 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-1c14590e-b8d6-4125-89a7-250c6e71a6dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6227bd78716a4a6684fe0672ecff35f5'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:58.957699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9c6ff68d-91dd-445f-bbdd-4f139d60aedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:58.974899 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-1c14590e-b8d6-4125-89a7-250c6e71a6dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6227bd78716a4a6684fe0672ecff35f5'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:59.004526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 841/1689] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:31:58 2022] DELETE /identity/v3/users/6227bd78716a4a6684fe0672ecff35f5 => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:31:59.006228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3721364-2c6e-4597-9c8a-3a021e578e2d None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:59.007657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3721364-2c6e-4597-9c8a-3a021e578e2d None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:59.008139 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3721364-2c6e-4597-9c8a-3a021e578e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:59.068858 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-b3721364-2c6e-4597-9c8a-3a021e578e2d None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:59.075797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b3721364-2c6e-4597-9c8a-3a021e578e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:59.197203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ff68d-91dd-445f-bbdd-4f139d60aedf None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:59.263226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 842/1690] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:31:59.263226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3721364-2c6e-4597-9c8a-3a021e578e2d None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:31:59.284859 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b3721364-2c6e-4597-9c8a-3a021e578e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:59.418525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:31:59.455429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3721364-2c6e-4597-9c8a-3a021e578e2d None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:31:59.466402 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5c425-5086-4a58-aa5d-1cda41aea2d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:59.467924 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5c425-5086-4a58-aa5d-1cda41aea2d4 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:59.468421 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5c425-5086-4a58-aa5d-1cda41aea2d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:59.468904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5c425-5086-4a58-aa5d-1cda41aea2d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:59.471188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-cba5c425-5086-4a58-aa5d-1cda41aea2d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:59.511809 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c888731d-ec93-4ce9-bee2-586ef9b86892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:31:59.516149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888731d-ec93-4ce9-bee2-586ef9b86892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:59.516443 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888731d-ec93-4ce9-bee2-586ef9b86892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:59.516728 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888731d-ec93-4ce9-bee2-586ef9b86892 admin admin] PATH_INFO: `/v3/projects/99f5f8dca0134df7b339987c309d2382` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:59.521861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c888731d-ec93-4ce9-bee2-586ef9b86892 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99f5f8dca0134df7b339987c309d2382)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:59.533635 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c888731d-ec93-4ce9-bee2-586ef9b86892 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:59.579111 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5c425-5086-4a58-aa5d-1cda41aea2d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:31:59.582478 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5c425-5086-4a58-aa5d-1cda41aea2d4 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:31:59.646813 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 843/1691] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:31:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:31:59.657406 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7905e9f-7532-4e0c-8929-d90421cdb0ac None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:31:59.663693 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7905e9f-7532-4e0c-8929-d90421cdb0ac None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:31:59.663693 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7905e9f-7532-4e0c-8929-d90421cdb0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:31:59.756477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-b7905e9f-7532-4e0c-8929-d90421cdb0ac None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:31:59.758495 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b7905e9f-7532-4e0c-8929-d90421cdb0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:31:59.784937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c888731d-ec93-4ce9-bee2-586ef9b86892 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99f5f8dca0134df7b339987c309d2382', 'request_id': 'req-c888731d-ec93-4ce9-bee2-586ef9b86892'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:59.930298 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c888731d-ec93-4ce9-bee2-586ef9b86892 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99f5f8dca0134df7b339987c309d2382', 'request_id': 'req-c888731d-ec93-4ce9-bee2-586ef9b86892'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:31:59.947588 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c888731d-ec93-4ce9-bee2-586ef9b86892 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99f5f8dca0134df7b339987c309d2382', 'request_id': 'req-c888731d-ec93-4ce9-bee2-586ef9b86892'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:00.007268 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c888731d-ec93-4ce9-bee2-586ef9b86892 admin admin] The token cache is being invalidate because project 99f5f8dca0134df7b339987c309d2382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:00.007607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c888731d-ec93-4ce9-bee2-586ef9b86892 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:00.010834 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c888731d-ec93-4ce9-bee2-586ef9b86892 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:00.021102 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 849/1692] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:31:58 2022] DELETE /identity/v3/projects/99f5f8dca0134df7b339987c309d2382 => generated 0 bytes in 1094 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:00.025521 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c888731d-ec93-4ce9-bee2-586ef9b86892 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:00.033008 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c888731d-ec93-4ce9-bee2-586ef9b86892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:00.149734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 844/1693] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:00.149734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba401-f0ad-4691-b74a-a86cd6bc8fec None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:00.150789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba401-f0ad-4691-b74a-a86cd6bc8fec None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:00.151201 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba401-f0ad-4691-b74a-a86cd6bc8fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:00.181689 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-8a3ba401-f0ad-4691-b74a-a86cd6bc8fec None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:00.193664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8a3ba401-f0ad-4691-b74a-a86cd6bc8fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:00.233815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 845/1694] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:00.242366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ba401-f0ad-4691-b74a-a86cd6bc8fec None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:00.253891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8a3ba401-f0ad-4691-b74a-a86cd6bc8fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:00.331323 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c888731d-ec93-4ce9-bee2-586ef9b86892 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:00.358783 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb8b66-520e-4d5f-8a01-d359a1cc1271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:00.359545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb8b66-520e-4d5f-8a01-d359a1cc1271 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:00.359843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb8b66-520e-4d5f-8a01-d359a1cc1271 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:00.362005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb8b66-520e-4d5f-8a01-d359a1cc1271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:00.375627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-64eb8b66-520e-4d5f-8a01-d359a1cc1271 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:00.413257 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb8b66-520e-4d5f-8a01-d359a1cc1271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:00.414944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb8b66-520e-4d5f-8a01-d359a1cc1271 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:00.438887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 850/1695] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:00.458103 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ba401-f0ad-4691-b74a-a86cd6bc8fec None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:00.462230 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91456f-6f31-4547-ac34-8285bf287557 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:00.463625 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91456f-6f31-4547-ac34-8285bf287557 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:00.463969 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91456f-6f31-4547-ac34-8285bf287557 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:00.464306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91456f-6f31-4547-ac34-8285bf287557 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:00.476201 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91456f-6f31-4547-ac34-8285bf287557 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:00.478003 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91456f-6f31-4547-ac34-8285bf287557 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:00.483778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca735b-7ca3-48b8-8ed9-9cebae45ba53 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:00.485983 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca735b-7ca3-48b8-8ed9-9cebae45ba53 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:00.489424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca735b-7ca3-48b8-8ed9-9cebae45ba53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:00.501561 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 846/1696] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:00.529632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d7471-0500-447b-af8f-4bb4c1355fab None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:00.530068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d7471-0500-447b-af8f-4bb4c1355fab None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:00.530509 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d7471-0500-447b-af8f-4bb4c1355fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:00.530857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-82ca735b-7ca3-48b8-8ed9-9cebae45ba53 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:00.533321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-82ca735b-7ca3-48b8-8ed9-9cebae45ba53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:00.557444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 851/1697] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:00.561904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca735b-7ca3-48b8-8ed9-9cebae45ba53 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:00.600178 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca735b-7ca3-48b8-8ed9-9cebae45ba53 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:00.600444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-118d7471-0500-447b-af8f-4bb4c1355fab None None] MFA Rules not processed for user `8e75c180bae74be0b2ccad079f92d32b`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:00.601928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-118d7471-0500-447b-af8f-4bb4c1355fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:00.617182 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399553b7-620e-4f56-98f7-00b28852e145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:00.620621 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399553b7-620e-4f56-98f7-00b28852e145 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:00.620621 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399553b7-620e-4f56-98f7-00b28852e145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:00.620621 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399553b7-620e-4f56-98f7-00b28852e145 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:00.620621 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399553b7-620e-4f56-98f7-00b28852e145 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:32:00.620621 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399553b7-620e-4f56-98f7-00b28852e145 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:00.621818 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399553b7-620e-4f56-98f7-00b28852e145 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:00.655947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 852/1698] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:32:00.664321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399553b7-620e-4f56-98f7-00b28852e145 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:00.681633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399553b7-620e-4f56-98f7-00b28852e145 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:00.710121 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e355ec-e84c-4e99-b69e-236815e41ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:00.711122 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e355ec-e84c-4e99-b69e-236815e41ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:00.711452 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e355ec-e84c-4e99-b69e-236815e41ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:00.711997 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e355ec-e84c-4e99-b69e-236815e41ac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:00.712924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e355ec-e84c-4e99-b69e-236815e41ac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:00.719698 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e355ec-e84c-4e99-b69e-236815e41ac5 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:00.755330 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 847/1699] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:00.801422 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3cc17-aa2b-48e5-a18e-9b70e135ab45 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:00.802136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3cc17-aa2b-48e5-a18e-9b70e135ab45 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:00.803003 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3cc17-aa2b-48e5-a18e-9b70e135ab45 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:00.805063 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 848/1700] 10.222.0.78 () {54 vars in 912 bytes} [Wed Jul 20 15:32:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:32:00.815547 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0367361-55f9-4d7e-a64a-10602b9d1c6c None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:00.821752 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0367361-55f9-4d7e-a64a-10602b9d1c6c None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:00.821752 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0367361-55f9-4d7e-a64a-10602b9d1c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:00.850756 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-f0367361-55f9-4d7e-a64a-10602b9d1c6c None None] MFA Rules not processed for user `b57530f18d9c4d328f8014db736d8de9`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:00.855903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f0367361-55f9-4d7e-a64a-10602b9d1c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:00.890635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 849/1701] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Jul 20 15:32:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:32:00.890857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 853/1702] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:00 2022] POST /identity/v3/projects => generated 357 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:32:00.893595 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0367361-55f9-4d7e-a64a-10602b9d1c6c None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:00.901967 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e355ec-e84c-4e99-b69e-236815e41ac5 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:00.905135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b6e355ec-e84c-4e99-b69e-236815e41ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:00.909071 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:00.975411 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0367361-55f9-4d7e-a64a-10602b9d1c6c None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:01.120739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e355ec-e84c-4e99-b69e-236815e41ac5 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:01.143790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4135923-b7b5-4ebe-891b-56de69fa97eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:01.153572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4135923-b7b5-4ebe-891b-56de69fa97eb service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:01.157190 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4135923-b7b5-4ebe-891b-56de69fa97eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:01.159569 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4135923-b7b5-4ebe-891b-56de69fa97eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:01.201626 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4135923-b7b5-4ebe-891b-56de69fa97eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:01.204294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4135923-b7b5-4ebe-891b-56de69fa97eb service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:01.216143 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eef676-99c5-47b1-8f88-2fa6f77a80ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:01.218559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eef676-99c5-47b1-8f88-2fa6f77a80ff admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:01.218559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eef676-99c5-47b1-8f88-2fa6f77a80ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:01.218559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eef676-99c5-47b1-8f88-2fa6f77a80ff admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:01.219212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eef676-99c5-47b1-8f88-2fa6f77a80ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:01.223718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eef676-99c5-47b1-8f88-2fa6f77a80ff admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:01.251637 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 854/1703] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:32:01.379827 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 850/1704] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:00 2022] POST /identity/v3/users => generated 279 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:01.382097 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4135923-b7b5-4ebe-891b-56de69fa97eb service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:01.457705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4135923-b7b5-4ebe-891b-56de69fa97eb service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:01.479273 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6ea2b-dedc-4f88-bb36-3b8f7298c90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:01.480060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6ea2b-dedc-4f88-bb36-3b8f7298c90b admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:01.480351 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6ea2b-dedc-4f88-bb36-3b8f7298c90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:01.480688 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6ea2b-dedc-4f88-bb36-3b8f7298c90b admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:01.483102 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed6ea2b-dedc-4f88-bb36-3b8f7298c90b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:01.488803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed6ea2b-dedc-4f88-bb36-3b8f7298c90b admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:01.546866 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 855/1705] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:32:01.549619 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eef676-99c5-47b1-8f88-2fa6f77a80ff admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:01.587751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eef676-99c5-47b1-8f88-2fa6f77a80ff admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:01.597067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4afe1-e925-4350-b5b5-16b37a07ba6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:01.598134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4afe1-e925-4350-b5b5-16b37a07ba6f admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:01.598712 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4afe1-e925-4350-b5b5-16b37a07ba6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:01.599200 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4afe1-e925-4350-b5b5-16b37a07ba6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:01.612283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4afe1-e925-4350-b5b5-16b37a07ba6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:01.614796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4afe1-e925-4350-b5b5-16b37a07ba6f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:01.629952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 851/1706] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:01.640686 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6ea2b-dedc-4f88-bb36-3b8f7298c90b admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:01.658639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6ea2b-dedc-4f88-bb36-3b8f7298c90b admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:01.680350 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435ec22-2d61-472b-904c-ed5cf7dc5804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:01.684606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435ec22-2d61-472b-904c-ed5cf7dc5804 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:01.685544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435ec22-2d61-472b-904c-ed5cf7dc5804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:01.687588 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435ec22-2d61-472b-904c-ed5cf7dc5804 admin admin] PATH_INFO: `/v3/projects/9994bd5e2ff64edc91b18b8c329da824/users/e950c6b6bbb34b9697e3a13585b29b7c/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:01.823256 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435ec22-2d61-472b-904c-ed5cf7dc5804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9994bd5e2ff64edc91b18b8c329da824, user_id=e950c6b6bbb34b9697e3a13585b29b7c, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:01.843839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435ec22-2d61-472b-904c-ed5cf7dc5804 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:01.921511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4afe1-e925-4350-b5b5-16b37a07ba6f admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:01.921511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f1a4afe1-e925-4350-b5b5-16b37a07ba6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:01.944935 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4afe1-e925-4350-b5b5-16b37a07ba6f admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:01.962077 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 856/1707] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:01 2022] PUT /identity/v3/projects/9994bd5e2ff64edc91b18b8c329da824/users/e950c6b6bbb34b9697e3a13585b29b7c/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:01.974651 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec73f8-41ce-41ae-92f1-fea26f2d47cf None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:01.975072 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec73f8-41ce-41ae-92f1-fea26f2d47cf None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:01.975617 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec73f8-41ce-41ae-92f1-fea26f2d47cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:02.039899 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-0fec73f8-41ce-41ae-92f1-fea26f2d47cf None None] MFA Rules not processed for user `e950c6b6bbb34b9697e3a13585b29b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:02.042536 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0fec73f8-41ce-41ae-92f1-fea26f2d47cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:02.080117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb863d-92f5-4662-8bca-a232627902a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:02.081040 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb863d-92f5-4662-8bca-a232627902a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:02.081353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb863d-92f5-4662-8bca-a232627902a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:02.081749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb863d-92f5-4662-8bca-a232627902a2 admin admin] PATH_INFO: `/v3/projects/bd335124dde5462b970264a96448f422` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:02.083361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb863d-92f5-4662-8bca-a232627902a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd335124dde5462b970264a96448f422)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:02.085897 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: WARNING py.warnings [None req-6dbb863d-92f5-4662-8bca-a232627902a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 15:32:02.085897 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: warnings.warn(msg) Jul 20 15:32:02.085897 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: Jul 20 15:32:02.087764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb863d-92f5-4662-8bca-a232627902a2 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:02.116864 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:02.182645 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 857/1708] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:01 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:32:02.192339 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec73f8-41ce-41ae-92f1-fea26f2d47cf None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:02.252939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec73f8-41ce-41ae-92f1-fea26f2d47cf None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:02.295082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabe7c8-862d-40e8-b5bf-ffa93c74674f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:02.296204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabe7c8-862d-40e8-b5bf-ffa93c74674f admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:02.302383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabe7c8-862d-40e8-b5bf-ffa93c74674f admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:02.302913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabe7c8-862d-40e8-b5bf-ffa93c74674f admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:02.304325 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabe7c8-862d-40e8-b5bf-ffa93c74674f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:02.329044 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabe7c8-862d-40e8-b5bf-ffa93c74674f admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:02.367452 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-6dbb863d-92f5-4662-8bca-a232627902a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd335124dde5462b970264a96448f422', 'request_id': 'req-6dbb863d-92f5-4662-8bca-a232627902a2'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:02.449474 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-6dbb863d-92f5-4662-8bca-a232627902a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd335124dde5462b970264a96448f422', 'request_id': 'req-6dbb863d-92f5-4662-8bca-a232627902a2'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:02.555954 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 858/1709] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:02 2022] POST /identity/v3/projects => generated 355 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:32:02.555954 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b41edd-7eac-4adb-9b4b-0f1c49d6dbe4 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:02.558028 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b41edd-7eac-4adb-9b4b-0f1c49d6dbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:02.558691 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b41edd-7eac-4adb-9b4b-0f1c49d6dbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:02.565844 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-6dbb863d-92f5-4662-8bca-a232627902a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd335124dde5462b970264a96448f422', 'request_id': 'req-6dbb863d-92f5-4662-8bca-a232627902a2'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:02.594794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:02.601641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-6dbb863d-92f5-4662-8bca-a232627902a2 admin admin] The token cache is being invalidate because project bd335124dde5462b970264a96448f422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:02.602379 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-6dbb863d-92f5-4662-8bca-a232627902a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:02.607543 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-6dbb863d-92f5-4662-8bca-a232627902a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:02.612046 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 852/1710] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:01 2022] DELETE /identity/v3/projects/bd335124dde5462b970264a96448f422 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:02.613456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb863d-92f5-4662-8bca-a232627902a2 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:02.623735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-6dbb863d-92f5-4662-8bca-a232627902a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:02.824877 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-f2b41edd-7eac-4adb-9b4b-0f1c49d6dbe4 None None] MFA Rules not processed for user `4e75a82cc7e94702ba0080fc65d6eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:02.829494 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f2b41edd-7eac-4adb-9b4b-0f1c49d6dbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:02.840619 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb863d-92f5-4662-8bca-a232627902a2 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:02.969443 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:02.992933 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd6ade-a8d1-407f-89bb-64d76498ac74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:02.997671 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd6ade-a8d1-407f-89bb-64d76498ac74 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:02.998250 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd6ade-a8d1-407f-89bb-64d76498ac74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:03.002705 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd6ade-a8d1-407f-89bb-64d76498ac74 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:03.002705 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd6ade-a8d1-407f-89bb-64d76498ac74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:03.004010 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd6ade-a8d1-407f-89bb-64d76498ac74 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:03.153359 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 859/1711] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Jul 20 15:32:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 600 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:03.156356 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b41edd-7eac-4adb-9b4b-0f1c49d6dbe4 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:03.164626 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f2b41edd-7eac-4adb-9b4b-0f1c49d6dbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:03.167199 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 853/1712] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:02 2022] POST /identity/v3/users => generated 284 bytes in 554 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:03.169278 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd6ade-a8d1-407f-89bb-64d76498ac74 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:03.184508 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd6ade-a8d1-407f-89bb-64d76498ac74 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:03.205183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f349dce-22d6-431c-972d-ffe1519b7d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:03.206978 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f349dce-22d6-431c-972d-ffe1519b7d00 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:03.209195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f349dce-22d6-431c-972d-ffe1519b7d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:03.209565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f349dce-22d6-431c-972d-ffe1519b7d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:03.212741 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f349dce-22d6-431c-972d-ffe1519b7d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:03.215948 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f349dce-22d6-431c-972d-ffe1519b7d00 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:03.229730 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b41edd-7eac-4adb-9b4b-0f1c49d6dbe4 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:03.242476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299c152-e819-4b33-9bf3-8228a517c084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:03.244882 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299c152-e819-4b33-9bf3-8228a517c084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:03.246478 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299c152-e819-4b33-9bf3-8228a517c084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:03.246943 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299c152-e819-4b33-9bf3-8228a517c084 admin admin] PATH_INFO: `/v3/users/0052a6855c594ed7a24450b41d0a5bc9` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:03.249849 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0299c152-e819-4b33-9bf3-8228a517c084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0052a6855c594ed7a24450b41d0a5bc9)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:03.250349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 854/1713] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:03.251396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f349dce-22d6-431c-972d-ffe1519b7d00 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:03.254929 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0299c152-e819-4b33-9bf3-8228a517c084 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:03.278551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f349dce-22d6-431c-972d-ffe1519b7d00 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:03.296089 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffba5be-de1a-46ab-9d7f-14817b161892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:03.297669 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffba5be-de1a-46ab-9d7f-14817b161892 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:03.299096 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffba5be-de1a-46ab-9d7f-14817b161892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:03.300254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffba5be-de1a-46ab-9d7f-14817b161892 admin admin] PATH_INFO: `/v3/projects/e826253d494d4bbf88f60658ea0e13b6/users/dd7340eca51346a19ffe1fada6bf7389/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:03.336855 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffba5be-de1a-46ab-9d7f-14817b161892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e826253d494d4bbf88f60658ea0e13b6, user_id=dd7340eca51346a19ffe1fada6bf7389, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:03.340569 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffba5be-de1a-46ab-9d7f-14817b161892 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:03.546476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 855/1714] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:03 2022] PUT /identity/v3/projects/e826253d494d4bbf88f60658ea0e13b6/users/dd7340eca51346a19ffe1fada6bf7389/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:03.553564 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffba5be-de1a-46ab-9d7f-14817b161892 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:03.620206 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0299c152-e819-4b33-9bf3-8228a517c084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0052a6855c594ed7a24450b41d0a5bc9'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:03.743984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffba5be-de1a-46ab-9d7f-14817b161892 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:03.766011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb2954-c430-41c2-9d00-10dc42bc180a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:03.768961 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb2954-c430-41c2-9d00-10dc42bc180a admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:03.769801 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb2954-c430-41c2-9d00-10dc42bc180a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:03.770307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb2954-c430-41c2-9d00-10dc42bc180a admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:03.771623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb2954-c430-41c2-9d00-10dc42bc180a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:03.773830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb2954-c430-41c2-9d00-10dc42bc180a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:03.782806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0299c152-e819-4b33-9bf3-8228a517c084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0052a6855c594ed7a24450b41d0a5bc9'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:03.788353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 856/1715] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:03.798126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb2954-c430-41c2-9d00-10dc42bc180a admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:03.801891 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0299c152-e819-4b33-9bf3-8228a517c084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0052a6855c594ed7a24450b41d0a5bc9'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:03.803578 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0299c152-e819-4b33-9bf3-8228a517c084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0052a6855c594ed7a24450b41d0a5bc9'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:03.824446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0299c152-e819-4b33-9bf3-8228a517c084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0052a6855c594ed7a24450b41d0a5bc9'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:03.838031 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 860/1716] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:03 2022] DELETE /identity/v3/users/0052a6855c594ed7a24450b41d0a5bc9 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:03.848393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299c152-e819-4b33-9bf3-8228a517c084 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:03.848393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0299c152-e819-4b33-9bf3-8228a517c084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:03.864745 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb2954-c430-41c2-9d00-10dc42bc180a admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:04.050739 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba144ae1-7a14-4595-9629-e447c9a43d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:04.051917 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba144ae1-7a14-4595-9629-e447c9a43d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:04.052376 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba144ae1-7a14-4595-9629-e447c9a43d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:04.052965 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba144ae1-7a14-4595-9629-e447c9a43d9c admin admin] PATH_INFO: `/v3/projects/e826253d494d4bbf88f60658ea0e13b6/users/dd7340eca51346a19ffe1fada6bf7389/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:04.060279 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba144ae1-7a14-4595-9629-e447c9a43d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e826253d494d4bbf88f60658ea0e13b6, user_id=dd7340eca51346a19ffe1fada6bf7389, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:04.066604 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba144ae1-7a14-4595-9629-e447c9a43d9c admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:04.107639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 857/1717] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:03 2022] PUT /identity/v3/projects/e826253d494d4bbf88f60658ea0e13b6/users/dd7340eca51346a19ffe1fada6bf7389/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:04.109337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ab686-076d-49c2-ae1e-a8a02d207323 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:04.109637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ab686-076d-49c2-ae1e-a8a02d207323 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:04.109920 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ab686-076d-49c2-ae1e-a8a02d207323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:04.184578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-f47ab686-076d-49c2-ae1e-a8a02d207323 None None] MFA Rules not processed for user `dd7340eca51346a19ffe1fada6bf7389`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:04.186381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f47ab686-076d-49c2-ae1e-a8a02d207323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:04.263009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299c152-e819-4b33-9bf3-8228a517c084 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:04.391165 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 858/1718] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:04 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:04.411108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ab686-076d-49c2-ae1e-a8a02d207323 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:04.413543 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f47ab686-076d-49c2-ae1e-a8a02d207323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:04.491685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e1c3d-3082-4c05-84e5-f27c16e73b62 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0d719c0eb04a3b8a228c6ce70fdf98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:04.493679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e1c3d-3082-4c05-84e5-f27c16e73b62 service swift] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:04.494476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e1c3d-3082-4c05-84e5-f27c16e73b62 service swift] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:04.495556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e1c3d-3082-4c05-84e5-f27c16e73b62 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:04.515171 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-357e1c3d-3082-4c05-84e5-f27c16e73b62 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:04.554835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ab686-076d-49c2-ae1e-a8a02d207323 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:04.564117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca976cba-0103-4991-8d86-f1925ee5a456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:04.564795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca976cba-0103-4991-8d86-f1925ee5a456 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:04.565084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca976cba-0103-4991-8d86-f1925ee5a456 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:04.565366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca976cba-0103-4991-8d86-f1925ee5a456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:04.595812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca976cba-0103-4991-8d86-f1925ee5a456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:04.609812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca976cba-0103-4991-8d86-f1925ee5a456 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:04.640921 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 859/1719] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:04 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:32:04.642410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca976cba-0103-4991-8d86-f1925ee5a456 service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:04.651429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ca976cba-0103-4991-8d86-f1925ee5a456 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:04.797713 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e1c3d-3082-4c05-84e5-f27c16e73b62 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:04.804343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e1c3d-3082-4c05-84e5-f27c16e73b62 service swift] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:04.846869 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 861/1720] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Jul 20 15:32:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1008 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:32:04.851297 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca976cba-0103-4991-8d86-f1925ee5a456 service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:04.876590 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035753d0-2f96-4dea-a62f-d6ed975f6a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:04.878503 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035753d0-2f96-4dea-a62f-d6ed975f6a32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:04.879584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035753d0-2f96-4dea-a62f-d6ed975f6a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:04.880839 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035753d0-2f96-4dea-a62f-d6ed975f6a32 admin admin] PATH_INFO: `/v3/projects/493fb99c808d45e186749e61845c48dc` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:04.884288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035753d0-2f96-4dea-a62f-d6ed975f6a32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=493fb99c808d45e186749e61845c48dc)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:04.894240 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035753d0-2f96-4dea-a62f-d6ed975f6a32 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:05.173066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e1c3d-3082-4c05-84e5-f27c16e73b62 service swift] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:05.198732 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e1c3d-3082-4c05-84e5-f27c16e73b62 service swift] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:05.224562 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:05.227217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-035753d0-2f96-4dea-a62f-d6ed975f6a32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '493fb99c808d45e186749e61845c48dc', 'request_id': 'req-035753d0-2f96-4dea-a62f-d6ed975f6a32'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:05.397863 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-035753d0-2f96-4dea-a62f-d6ed975f6a32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '493fb99c808d45e186749e61845c48dc', 'request_id': 'req-035753d0-2f96-4dea-a62f-d6ed975f6a32'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:05.420823 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-035753d0-2f96-4dea-a62f-d6ed975f6a32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '493fb99c808d45e186749e61845c48dc', 'request_id': 'req-035753d0-2f96-4dea-a62f-d6ed975f6a32'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:05.474978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:05.484111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75164211-d621-4183-b558-323c7ec43e76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:05.485349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75164211-d621-4183-b558-323c7ec43e76 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:05.485888 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75164211-d621-4183-b558-323c7ec43e76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:05.486606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75164211-d621-4183-b558-323c7ec43e76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:05.495690 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-75164211-d621-4183-b558-323c7ec43e76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:05.511251 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-035753d0-2f96-4dea-a62f-d6ed975f6a32 admin admin] The token cache is being invalidate because project 493fb99c808d45e186749e61845c48dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:05.513469 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-035753d0-2f96-4dea-a62f-d6ed975f6a32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:05.517572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-035753d0-2f96-4dea-a62f-d6ed975f6a32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:05.525437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 860/1721] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:04 2022] DELETE /identity/v3/projects/493fb99c808d45e186749e61845c48dc => generated 0 bytes in 884 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:05.533490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a22e7-275c-46fc-b882-5ef908ce493a None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:05.534206 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a22e7-275c-46fc-b882-5ef908ce493a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:05.534677 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a22e7-275c-46fc-b882-5ef908ce493a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:05.638029 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-1f4a22e7-275c-46fc-b882-5ef908ce493a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:05.644820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-1f4a22e7-275c-46fc-b882-5ef908ce493a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:05.882290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75164211-d621-4183-b558-323c7ec43e76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:05.900168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75164211-d621-4183-b558-323c7ec43e76 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:05.978281 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 862/1722] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:05 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 806 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:32:06.049879 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 861/1723] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 518 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:06.080205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28711f97-4142-4580-be85-cc99b5dd81a0 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:06.084609 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28711f97-4142-4580-be85-cc99b5dd81a0 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:06.086119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28711f97-4142-4580-be85-cc99b5dd81a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:06.308672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-28711f97-4142-4580-be85-cc99b5dd81a0 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:06.328323 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-28711f97-4142-4580-be85-cc99b5dd81a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:06.421999 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a22e7-275c-46fc-b882-5ef908ce493a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:06.431898 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 863/1724] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:06.479681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a22e7-275c-46fc-b882-5ef908ce493a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:06.513603 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555cd998-ad93-4401-97a1-7f63f93a7293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:06.514824 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cd998-ad93-4401-97a1-7f63f93a7293 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:06.515540 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cd998-ad93-4401-97a1-7f63f93a7293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:06.516925 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cd998-ad93-4401-97a1-7f63f93a7293 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:06.517982 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555cd998-ad93-4401-97a1-7f63f93a7293 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:32:06.519931 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555cd998-ad93-4401-97a1-7f63f93a7293 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:06.535141 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555cd998-ad93-4401-97a1-7f63f93a7293 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:06.596334 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 862/1725] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:32:06.599668 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28711f97-4142-4580-be85-cc99b5dd81a0 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:06.653126 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28711f97-4142-4580-be85-cc99b5dd81a0 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:06.691176 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c0ded-e39b-4da8-98bd-7bc5db3ed7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:06.693258 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c0ded-e39b-4da8-98bd-7bc5db3ed7a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:06.693861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c0ded-e39b-4da8-98bd-7bc5db3ed7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:06.694552 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c0ded-e39b-4da8-98bd-7bc5db3ed7a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:06.696202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c0ded-e39b-4da8-98bd-7bc5db3ed7a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:06.700775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c0ded-e39b-4da8-98bd-7bc5db3ed7a2 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:06.906895 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce1969-173e-457c-9f61-fcb3961b14ad None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:06.906895 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce1969-173e-457c-9f61-fcb3961b14ad None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:06.906895 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce1969-173e-457c-9f61-fcb3961b14ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:06.936096 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 864/1726] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:06 2022] POST /identity/v3/projects => generated 379 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:06.942388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c0ded-e39b-4da8-98bd-7bc5db3ed7a2 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:06.944723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-7cce1969-173e-457c-9f61-fcb3961b14ad None None] MFA Rules not processed for user `b57530f18d9c4d328f8014db736d8de9`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:06.946013 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-7cce1969-173e-457c-9f61-fcb3961b14ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:07.242729 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 863/1727] 10.222.0.102 () {58 vars in 1016 bytes} [Wed Jul 20 15:32:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:07.408576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c0ded-e39b-4da8-98bd-7bc5db3ed7a2 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:07.439018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409770f7-5294-473d-a0b7-7c94a1674200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:07.440020 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409770f7-5294-473d-a0b7-7c94a1674200 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:07.440491 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409770f7-5294-473d-a0b7-7c94a1674200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:07.440975 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409770f7-5294-473d-a0b7-7c94a1674200 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:07.442186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409770f7-5294-473d-a0b7-7c94a1674200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:07.448549 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409770f7-5294-473d-a0b7-7c94a1674200 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:07.464952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce1969-173e-457c-9f61-fcb3961b14ad None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:07.466497 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-7cce1969-173e-457c-9f61-fcb3961b14ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:07.515871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce1969-173e-457c-9f61-fcb3961b14ad None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:07.524019 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c93b86-9167-4af5-988b-0312d0cff7d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:07.525501 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c93b86-9167-4af5-988b-0312d0cff7d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:07.525950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c93b86-9167-4af5-988b-0312d0cff7d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:07.526417 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c93b86-9167-4af5-988b-0312d0cff7d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:07.552983 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c93b86-9167-4af5-988b-0312d0cff7d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:07.555200 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c93b86-9167-4af5-988b-0312d0cff7d6 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:07.595472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 864/1728] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:07.623731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 865/1729] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:06 2022] POST /identity/v3/users => generated 290 bytes in 682 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:32:07.628963 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c93b86-9167-4af5-988b-0312d0cff7d6 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:07.650180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c93b86-9167-4af5-988b-0312d0cff7d6 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:07.668873 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787345ff-0eae-452f-b2a6-871b201787f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:07.669575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787345ff-0eae-452f-b2a6-871b201787f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:07.669871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787345ff-0eae-452f-b2a6-871b201787f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:07.670157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787345ff-0eae-452f-b2a6-871b201787f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:07.670986 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787345ff-0eae-452f-b2a6-871b201787f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:07.673416 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787345ff-0eae-452f-b2a6-871b201787f9 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:07.696758 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 865/1730] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:07.702170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409770f7-5294-473d-a0b7-7c94a1674200 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:07.720904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409770f7-5294-473d-a0b7-7c94a1674200 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:07.738307 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6557e4-8000-4646-9d9d-076f524868cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:07.744861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6557e4-8000-4646-9d9d-076f524868cb admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:07.744861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6557e4-8000-4646-9d9d-076f524868cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:07.744861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6557e4-8000-4646-9d9d-076f524868cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:07.744861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6557e4-8000-4646-9d9d-076f524868cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:07.747406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6557e4-8000-4646-9d9d-076f524868cb admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:07.781467 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 866/1731] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:07.783032 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787345ff-0eae-452f-b2a6-871b201787f9 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:07.802155 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787345ff-0eae-452f-b2a6-871b201787f9 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:07.819107 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a7a85-b4d4-4885-91ad-ec3bef5819f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:07.820362 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a7a85-b4d4-4885-91ad-ec3bef5819f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:07.820812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a7a85-b4d4-4885-91ad-ec3bef5819f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:07.821245 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a7a85-b4d4-4885-91ad-ec3bef5819f5 admin admin] PATH_INFO: `/v3/projects/0df3a7f0776c411db4604ac7bfe5719a/users/d05979493394438dadc186c47c590457/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:07.860597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a7a85-b4d4-4885-91ad-ec3bef5819f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df3a7f0776c411db4604ac7bfe5719a, user_id=d05979493394438dadc186c47c590457, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:07.864283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a7a85-b4d4-4885-91ad-ec3bef5819f5 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:08.072866 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 866/1732] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:07 2022] PUT /identity/v3/projects/0df3a7f0776c411db4604ac7bfe5719a/users/d05979493394438dadc186c47c590457/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:08.100377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56713a2c-5a96-4621-aeb6-9141bd416734 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:08.104578 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56713a2c-5a96-4621-aeb6-9141bd416734 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:08.104578 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56713a2c-5a96-4621-aeb6-9141bd416734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:08.220932 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:08.240404 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-56713a2c-5a96-4621-aeb6-9141bd416734 None None] MFA Rules not processed for user `d05979493394438dadc186c47c590457`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:08.258622 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-56713a2c-5a96-4621-aeb6-9141bd416734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:08.461095 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 867/1733] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:08.467819 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a7a85-b4d4-4885-91ad-ec3bef5819f5 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:08.577548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a7a85-b4d4-4885-91ad-ec3bef5819f5 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:08.587857 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3656db-a05a-4bbe-991b-d13c64d3b228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:08.589242 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3656db-a05a-4bbe-991b-d13c64d3b228 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:08.589740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3656db-a05a-4bbe-991b-d13c64d3b228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:08.590422 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3656db-a05a-4bbe-991b-d13c64d3b228 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:08.594049 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3656db-a05a-4bbe-991b-d13c64d3b228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:08.594049 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3656db-a05a-4bbe-991b-d13c64d3b228 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:08.722430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 867/1734] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:08 2022] POST /identity/v3/projects => generated 379 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:08.730349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56713a2c-5a96-4621-aeb6-9141bd416734 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:08.755043 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56713a2c-5a96-4621-aeb6-9141bd416734 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:08.764481 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:08.933716 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fcb18-25e9-4993-9b1b-b7c6fc1a10f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:08.934749 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fcb18-25e9-4993-9b1b-b7c6fc1a10f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:08.935200 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fcb18-25e9-4993-9b1b-b7c6fc1a10f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:08.935645 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fcb18-25e9-4993-9b1b-b7c6fc1a10f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:08.936772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fcb18-25e9-4993-9b1b-b7c6fc1a10f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:08.939338 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fcb18-25e9-4993-9b1b-b7c6fc1a10f6 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:09.073930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 868/1735] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:08 2022] POST /identity/v3/users => generated 296 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:09.082962 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3656db-a05a-4bbe-991b-d13c64d3b228 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:09.172488 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3656db-a05a-4bbe-991b-d13c64d3b228 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:09.194557 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10d704-fbe3-4425-a863-6d5f71843c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:09.196187 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10d704-fbe3-4425-a863-6d5f71843c06 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:09.196187 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10d704-fbe3-4425-a863-6d5f71843c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:09.201463 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10d704-fbe3-4425-a863-6d5f71843c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:09.203313 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10d704-fbe3-4425-a863-6d5f71843c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:09.218667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10d704-fbe3-4425-a863-6d5f71843c06 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:09.269128 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 868/1736] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:09.275853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fcb18-25e9-4993-9b1b-b7c6fc1a10f6 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:09.295972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fcb18-25e9-4993-9b1b-b7c6fc1a10f6 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:09.309315 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7837f30-e5e2-4de6-b2b3-4d72992a8df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:09.313117 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7837f30-e5e2-4de6-b2b3-4d72992a8df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:09.313438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7837f30-e5e2-4de6-b2b3-4d72992a8df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:09.313732 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7837f30-e5e2-4de6-b2b3-4d72992a8df1 admin admin] PATH_INFO: `/v3/projects/a3651eaccb084707aeedcad163e7953b/users/733c17f3f48f45f697a70a2af9b7c902/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:09.406831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7837f30-e5e2-4de6-b2b3-4d72992a8df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3651eaccb084707aeedcad163e7953b, user_id=733c17f3f48f45f697a70a2af9b7c902, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:09.409963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7837f30-e5e2-4de6-b2b3-4d72992a8df1 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:09.920315 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 869/1737] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:09 2022] PUT /identity/v3/projects/a3651eaccb084707aeedcad163e7953b/users/733c17f3f48f45f697a70a2af9b7c902/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:09.928454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10d704-fbe3-4425-a863-6d5f71843c06 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:09.964155 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10d704-fbe3-4425-a863-6d5f71843c06 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:10.047365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7837f30-e5e2-4de6-b2b3-4d72992a8df1 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:10.049899 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c7837f30-e5e2-4de6-b2b3-4d72992a8df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:10.403773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f1e49-2264-491b-b066-754c9eec7f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:10.403773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f1e49-2264-491b-b066-754c9eec7f0a admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:10.403773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f1e49-2264-491b-b066-754c9eec7f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:10.403773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f1e49-2264-491b-b066-754c9eec7f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:10.404170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f1e49-2264-491b-b066-754c9eec7f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:10.404170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f1e49-2264-491b-b066-754c9eec7f0a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:10.504811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 869/1738] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 577 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:10.512528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f1e49-2264-491b-b066-754c9eec7f0a admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:10.611638 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f1e49-2264-491b-b066-754c9eec7f0a admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:10.636924 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab7bb5-5986-4bb3-88ba-97064ce28c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:10.640206 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab7bb5-5986-4bb3-88ba-97064ce28c02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:10.640922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab7bb5-5986-4bb3-88ba-97064ce28c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:10.641983 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab7bb5-5986-4bb3-88ba-97064ce28c02 admin admin] PATH_INFO: `/v3/projects/a3651eaccb084707aeedcad163e7953b/users/733c17f3f48f45f697a70a2af9b7c902/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:10.659831 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab7bb5-5986-4bb3-88ba-97064ce28c02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3651eaccb084707aeedcad163e7953b, user_id=733c17f3f48f45f697a70a2af9b7c902, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:10.670432 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab7bb5-5986-4bb3-88ba-97064ce28c02 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:10.691819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7837f30-e5e2-4de6-b2b3-4d72992a8df1 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:10.705418 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 870/1739] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:10 2022] PUT /identity/v3/projects/a3651eaccb084707aeedcad163e7953b/users/733c17f3f48f45f697a70a2af9b7c902/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:10.724701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e4371-5c52-48e0-8a28-466ad73e5af5 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:10.726505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e4371-5c52-48e0-8a28-466ad73e5af5 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:10.727691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e4371-5c52-48e0-8a28-466ad73e5af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:10.822609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-0b4e4371-5c52-48e0-8a28-466ad73e5af5 None None] MFA Rules not processed for user `733c17f3f48f45f697a70a2af9b7c902`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:10.827499 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0b4e4371-5c52-48e0-8a28-466ad73e5af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:10.926461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:10.935106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:10.938495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:10.938806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a admin admin] PATH_INFO: `/v3/projects/ea5333d99a3d4bf3acf91f096268f4b9` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:10.946788 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea5333d99a3d4bf3acf91f096268f4b9)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:10.949816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:11.119681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:11.231963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea5333d99a3d4bf3acf91f096268f4b9', 'request_id': 'req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:11.280776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 871/1740] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 558 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:11.409587 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea5333d99a3d4bf3acf91f096268f4b9', 'request_id': 'req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:11.461006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea5333d99a3d4bf3acf91f096268f4b9', 'request_id': 'req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:11.537157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a admin admin] The token cache is being invalidate because project ea5333d99a3d4bf3acf91f096268f4b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:11.537522 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:11.543076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:11.549411 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 870/1741] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:10 2022] DELETE /identity/v3/projects/ea5333d99a3d4bf3acf91f096268f4b9 => generated 0 bytes in 1503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:11.554675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309707e-85e7-4bad-9354-c6e70f31a6b0 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:11.555198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309707e-85e7-4bad-9354-c6e70f31a6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:11.556178 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309707e-85e7-4bad-9354-c6e70f31a6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:11.598085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-1309707e-85e7-4bad-9354-c6e70f31a6b0 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:11.599939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-1309707e-85e7-4bad-9354-c6e70f31a6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:11.610421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:11.611974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:11.754834 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:11.779719 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 872/1742] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:11.791854 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61597d13-a50f-4bb7-b545-62fce10df39f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:11.792269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61597d13-a50f-4bb7-b545-62fce10df39f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:11.792773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61597d13-a50f-4bb7-b545-62fce10df39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:11.801030 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603b1f3-3e7e-4b3e-8e58-99ac254ff01a admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:11.819489 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a09821-85d5-4a81-b5db-f3e09a0cb0c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:11.820476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a09821-85d5-4a81-b5db-f3e09a0cb0c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:11.821505 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a09821-85d5-4a81-b5db-f3e09a0cb0c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:11.825688 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a09821-85d5-4a81-b5db-f3e09a0cb0c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:11.840882 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-89a09821-85d5-4a81-b5db-f3e09a0cb0c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:11.866924 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-61597d13-a50f-4bb7-b545-62fce10df39f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:11.868231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-61597d13-a50f-4bb7-b545-62fce10df39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:11.905513 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 873/1743] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:32:11.909697 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61597d13-a50f-4bb7-b545-62fce10df39f None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:11.962756 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61597d13-a50f-4bb7-b545-62fce10df39f None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:12.018353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81add7-6fdc-4bb3-91e1-dead6dc36255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:12.019725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81add7-6fdc-4bb3-91e1-dead6dc36255 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:12.020015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81add7-6fdc-4bb3-91e1-dead6dc36255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:12.020517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81add7-6fdc-4bb3-91e1-dead6dc36255 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:12.022901 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81add7-6fdc-4bb3-91e1-dead6dc36255 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:32:12.031295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81add7-6fdc-4bb3-91e1-dead6dc36255 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:12.031295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81add7-6fdc-4bb3-91e1-dead6dc36255 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:12.098321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 874/1744] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:32:12.102115 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81add7-6fdc-4bb3-91e1-dead6dc36255 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:12.141796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81add7-6fdc-4bb3-91e1-dead6dc36255 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:12.147614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a09821-85d5-4a81-b5db-f3e09a0cb0c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:12.147614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a09821-85d5-4a81-b5db-f3e09a0cb0c7 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:12.171655 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec2c5f-ff7e-4c82-bcf7-be381a2ee68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:12.175295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec2c5f-ff7e-4c82-bcf7-be381a2ee68c admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:12.175581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec2c5f-ff7e-4c82-bcf7-be381a2ee68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:12.175863 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec2c5f-ff7e-4c82-bcf7-be381a2ee68c admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:12.176608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec2c5f-ff7e-4c82-bcf7-be381a2ee68c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:12.181795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec2c5f-ff7e-4c82-bcf7-be381a2ee68c admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:12.193135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 871/1745] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:12.346671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f49d17-6d29-41f7-a8c1-ca967bf742ba None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:12.346968 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f49d17-6d29-41f7-a8c1-ca967bf742ba None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:12.347525 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f49d17-6d29-41f7-a8c1-ca967bf742ba None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:12.349033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 872/1746] 10.222.0.78 () {54 vars in 921 bytes} [Wed Jul 20 15:32:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:32:12.460522 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f49d17-6d29-41f7-a8c1-ca967bf742ba None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:12.472896 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 875/1747] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:12 2022] POST /identity/v3/projects => generated 369 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:12.545256 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568b7fc-0cde-4467-ade1-52e17689e32a None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:12.545256 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568b7fc-0cde-4467-ade1-52e17689e32a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:12.545256 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568b7fc-0cde-4467-ade1-52e17689e32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:12.596913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-8568b7fc-0cde-4467-ade1-52e17689e32a None None] MFA Rules not processed for user `a4a24d5a375945779fa626fae25ac237`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:12.598357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8568b7fc-0cde-4467-ade1-52e17689e32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:12.670862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f49d17-6d29-41f7-a8c1-ca967bf742ba None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:12.674759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09674108-3fc1-45db-acb9-69e9fa841bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:12.675970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09674108-3fc1-45db-acb9-69e9fa841bcc admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:12.676514 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09674108-3fc1-45db-acb9-69e9fa841bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:12.686630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09674108-3fc1-45db-acb9-69e9fa841bcc admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:12.687807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09674108-3fc1-45db-acb9-69e9fa841bcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:12.690032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09674108-3fc1-45db-acb9-69e9fa841bcc admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:12.830007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 873/1748] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:12 2022] POST /identity/v3/users => generated 285 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:12.831783 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74947d3-0c50-4f9c-9190-c89e37bc2f25 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:12.832077 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74947d3-0c50-4f9c-9190-c89e37bc2f25 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:12.833760 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74947d3-0c50-4f9c-9190-c89e37bc2f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:12.866597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 876/1749] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Jul 20 15:32:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:32:12.867701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568b7fc-0cde-4467-ade1-52e17689e32a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:12.907233 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-e74947d3-0c50-4f9c-9190-c89e37bc2f25 None None] MFA Rules not processed for user `6498ef5c517c4fa4abed90e430936ded`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:12.915929 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e74947d3-0c50-4f9c-9190-c89e37bc2f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:13.002142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568b7fc-0cde-4467-ade1-52e17689e32a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:13.007538 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09364a-ee8a-4a5b-b4d3-66d91fe8baa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:13.008567 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09364a-ee8a-4a5b-b4d3-66d91fe8baa1 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:13.009524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09364a-ee8a-4a5b-b4d3-66d91fe8baa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:13.009963 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09364a-ee8a-4a5b-b4d3-66d91fe8baa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:13.012308 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-dc09364a-ee8a-4a5b-b4d3-66d91fe8baa1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:13.093358 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 874/1750] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:13.093358 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74947d3-0c50-4f9c-9190-c89e37bc2f25 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:13.118144 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74947d3-0c50-4f9c-9190-c89e37bc2f25 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:13.147405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090db5b2-952e-41dd-81a8-831555b085db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:13.148125 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090db5b2-952e-41dd-81a8-831555b085db admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:13.148767 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090db5b2-952e-41dd-81a8-831555b085db admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:13.151839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090db5b2-952e-41dd-81a8-831555b085db admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:13.153208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090db5b2-952e-41dd-81a8-831555b085db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:13.154376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090db5b2-952e-41dd-81a8-831555b085db admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:13.159649 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09364a-ee8a-4a5b-b4d3-66d91fe8baa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:13.163941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09364a-ee8a-4a5b-b4d3-66d91fe8baa1 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:13.194636 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 875/1751] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:13.195126 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090db5b2-952e-41dd-81a8-831555b085db admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:13.197350 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 877/1752] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:13.242468 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090db5b2-952e-41dd-81a8-831555b085db admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:13.266009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fcbb58-d8ee-474c-89fe-5950ecb06aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:13.266009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcbb58-d8ee-474c-89fe-5950ecb06aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:13.266009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcbb58-d8ee-474c-89fe-5950ecb06aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:13.266636 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcbb58-d8ee-474c-89fe-5950ecb06aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:13.268155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fcbb58-d8ee-474c-89fe-5950ecb06aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:13.270010 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fcbb58-d8ee-474c-89fe-5950ecb06aa1 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:13.305337 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:13.306619 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 876/1753] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:13.308957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09364a-ee8a-4a5b-b4d3-66d91fe8baa1 service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:13.321137 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09364a-ee8a-4a5b-b4d3-66d91fe8baa1 service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:13.328880 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ceba7d-7a36-49ed-8cea-69902c165cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:13.330618 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ceba7d-7a36-49ed-8cea-69902c165cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:13.331083 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ceba7d-7a36-49ed-8cea-69902c165cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:13.331599 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ceba7d-7a36-49ed-8cea-69902c165cd6 admin admin] PATH_INFO: `/v3/projects/a6c62bc1662845d2b66459d06df3d0eb/users/1c1dedef8e3b49de94aeae690b7c8419/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:13.358143 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ceba7d-7a36-49ed-8cea-69902c165cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6c62bc1662845d2b66459d06df3d0eb, user_id=1c1dedef8e3b49de94aeae690b7c8419, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:13.361240 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ceba7d-7a36-49ed-8cea-69902c165cd6 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:13.452822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fcbb58-d8ee-474c-89fe-5950ecb06aa1 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:13.454764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b5fcbb58-d8ee-474c-89fe-5950ecb06aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:13.490735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 878/1754] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:13 2022] PUT /identity/v3/projects/a6c62bc1662845d2b66459d06df3d0eb/users/1c1dedef8e3b49de94aeae690b7c8419/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:13.510685 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7220b8-a9a5-4e5a-9208-c19614bf6e44 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:13.510685 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7220b8-a9a5-4e5a-9208-c19614bf6e44 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:13.510685 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7220b8-a9a5-4e5a-9208-c19614bf6e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:13.520050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fcbb58-d8ee-474c-89fe-5950ecb06aa1 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:13.630331 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-bb7220b8-a9a5-4e5a-9208-c19614bf6e44 None None] MFA Rules not processed for user `1c1dedef8e3b49de94aeae690b7c8419`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:13.631608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-bb7220b8-a9a5-4e5a-9208-c19614bf6e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:13.839559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6942ce3-bfd4-40d3-9489-011335be3676 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:13.840540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6942ce3-bfd4-40d3-9489-011335be3676 service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:13.841005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6942ce3-bfd4-40d3-9489-011335be3676 service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:13.841438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6942ce3-bfd4-40d3-9489-011335be3676 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:13.867618 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6942ce3-bfd4-40d3-9489-011335be3676 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:13.869702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6942ce3-bfd4-40d3-9489-011335be3676 service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:13.903322 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 877/1755] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:32:13.948691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 879/1756] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:13 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:13.953359 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6942ce3-bfd4-40d3-9489-011335be3676 service nova] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:14.174748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6942ce3-bfd4-40d3-9489-011335be3676 service nova] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:14.198244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f54f2b-f04d-428b-9b6c-625ee691644d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:14.203384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f54f2b-f04d-428b-9b6c-625ee691644d admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:14.203384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f54f2b-f04d-428b-9b6c-625ee691644d admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:14.203384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f54f2b-f04d-428b-9b6c-625ee691644d admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:14.203384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f54f2b-f04d-428b-9b6c-625ee691644d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:14.203384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f54f2b-f04d-428b-9b6c-625ee691644d admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:14.429556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 878/1757] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:13 2022] POST /identity/v3/projects => generated 367 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:14.429993 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7220b8-a9a5-4e5a-9208-c19614bf6e44 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:14.447004 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7220b8-a9a5-4e5a-9208-c19614bf6e44 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:14.562296 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0ee87-0538-4224-97a3-3b9f2ff1fdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:14.562296 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0ee87-0538-4224-97a3-3b9f2ff1fdbd admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:14.562296 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0ee87-0538-4224-97a3-3b9f2ff1fdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:14.572483 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0ee87-0538-4224-97a3-3b9f2ff1fdbd admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:14.573320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0ee87-0538-4224-97a3-3b9f2ff1fdbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:14.575348 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0ee87-0538-4224-97a3-3b9f2ff1fdbd admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:14.797448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 880/1758] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:14 2022] POST /identity/v3/users => generated 290 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:14.798450 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f54f2b-f04d-428b-9b6c-625ee691644d admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:14.812998 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f54f2b-f04d-428b-9b6c-625ee691644d admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:14.828133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b47fa8-4f3e-46f2-8121-647be87639b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:14.829020 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b47fa8-4f3e-46f2-8121-647be87639b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:14.829473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b47fa8-4f3e-46f2-8121-647be87639b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:14.829998 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b47fa8-4f3e-46f2-8121-647be87639b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:14.832550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b47fa8-4f3e-46f2-8121-647be87639b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:14.837504 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b47fa8-4f3e-46f2-8121-647be87639b0 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:14.871158 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0ee87-0538-4224-97a3-3b9f2ff1fdbd admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:14.877954 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 879/1759] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:14.882096 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0ee87-0538-4224-97a3-3b9f2ff1fdbd admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:14.897609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfe70d-530a-43b0-984a-182db802cb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:14.899195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfe70d-530a-43b0-984a-182db802cb9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:14.899383 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfe70d-530a-43b0-984a-182db802cb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:14.899675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfe70d-530a-43b0-984a-182db802cb9c admin admin] PATH_INFO: `/v3/projects/bada7bf38324432682040e9c9cc04f30/users/cebfe156d4f54ebcac371ea13424b649/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:14.950416 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfe70d-530a-43b0-984a-182db802cb9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bada7bf38324432682040e9c9cc04f30, user_id=cebfe156d4f54ebcac371ea13424b649, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:14.953726 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfe70d-530a-43b0-984a-182db802cb9c admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:15.088253 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 881/1760] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:14 2022] PUT /identity/v3/projects/bada7bf38324432682040e9c9cc04f30/users/cebfe156d4f54ebcac371ea13424b649/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:15.100594 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b47fa8-4f3e-46f2-8121-647be87639b0 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:15.109329 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b47fa8-4f3e-46f2-8121-647be87639b0 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:15.257027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45b111-b9f2-4a39-a300-16b4df4eb488 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:15.257524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45b111-b9f2-4a39-a300-16b4df4eb488 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:15.257954 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45b111-b9f2-4a39-a300-16b4df4eb488 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:15.260720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 882/1761] 10.222.0.78 () {54 vars in 921 bytes} [Wed Jul 20 15:32:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:32:15.278724 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a1082-8792-451b-b753-c1c4ea1549a1 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:15.278724 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a1082-8792-451b-b753-c1c4ea1549a1 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:15.278724 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a1082-8792-451b-b753-c1c4ea1549a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:15.279888 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6482e-1b6f-4133-a459-4019e992e1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:15.284445 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6482e-1b6f-4133-a459-4019e992e1f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:15.286168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6482e-1b6f-4133-a459-4019e992e1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:15.286580 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6482e-1b6f-4133-a459-4019e992e1f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:15.287868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6482e-1b6f-4133-a459-4019e992e1f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:15.292862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6482e-1b6f-4133-a459-4019e992e1f6 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:15.355671 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:15.362657 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 880/1762] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:15.370148 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6482e-1b6f-4133-a459-4019e992e1f6 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:15.370403 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-5c0a1082-8792-451b-b753-c1c4ea1549a1 None None] MFA Rules not processed for user `a4a24d5a375945779fa626fae25ac237`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:15.373666 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5c0a1082-8792-451b-b753-c1c4ea1549a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:15.395750 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6482e-1b6f-4133-a459-4019e992e1f6 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:15.408832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3544dc-6f66-407e-b184-b37998fcd66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:15.408832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3544dc-6f66-407e-b184-b37998fcd66c admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:15.408832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3544dc-6f66-407e-b184-b37998fcd66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:15.408832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3544dc-6f66-407e-b184-b37998fcd66c admin admin] PATH_INFO: `/v3/projects/bada7bf38324432682040e9c9cc04f30/users/cebfe156d4f54ebcac371ea13424b649/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:15.437378 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3544dc-6f66-407e-b184-b37998fcd66c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bada7bf38324432682040e9c9cc04f30, user_id=cebfe156d4f54ebcac371ea13424b649, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:15.467056 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3544dc-6f66-407e-b184-b37998fcd66c admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:15.522887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 881/1763] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:15 2022] PUT /identity/v3/projects/bada7bf38324432682040e9c9cc04f30/users/cebfe156d4f54ebcac371ea13424b649/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:15.535701 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f3e14-c95c-49c8-8fc3-408c758186d4 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:15.537133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f3e14-c95c-49c8-8fc3-408c758186d4 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:15.539198 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f3e14-c95c-49c8-8fc3-408c758186d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:15.614374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 883/1764] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Jul 20 15:32:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:32:15.644301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-b88f3e14-c95c-49c8-8fc3-408c758186d4 None None] MFA Rules not processed for user `cebfe156d4f54ebcac371ea13424b649`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:15.646720 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b88f3e14-c95c-49c8-8fc3-408c758186d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:15.859076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 882/1765] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:15 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:15.872209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a1082-8792-451b-b753-c1c4ea1549a1 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:16.069656 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a1082-8792-451b-b753-c1c4ea1549a1 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:16.156447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5b420-67b0-4806-8a86-5818619ef5b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:16.157736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5b420-67b0-4806-8a86-5818619ef5b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:16.169133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5b420-67b0-4806-8a86-5818619ef5b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:16.169700 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5b420-67b0-4806-8a86-5818619ef5b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:16.188837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5b420-67b0-4806-8a86-5818619ef5b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:16.191008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5b420-67b0-4806-8a86-5818619ef5b4 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:16.223397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 884/1766] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:15 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:16.330675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f3e14-c95c-49c8-8fc3-408c758186d4 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:16.334250 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b88f3e14-c95c-49c8-8fc3-408c758186d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:16.554672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f3e14-c95c-49c8-8fc3-408c758186d4 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:16.572445 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db251c-94fd-4118-9c50-615a02a69373 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:16.575342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db251c-94fd-4118-9c50-615a02a69373 service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:16.575342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db251c-94fd-4118-9c50-615a02a69373 service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:16.575342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db251c-94fd-4118-9c50-615a02a69373 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:16.607612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:16.614921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db251c-94fd-4118-9c50-615a02a69373 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:16.621788 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db251c-94fd-4118-9c50-615a02a69373 service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:16.685591 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 883/1767] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:20.341849 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:21.186720 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:22.781212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:23.265488 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5b420-67b0-4806-8a86-5818619ef5b4 service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:23.327188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5b420-67b0-4806-8a86-5818619ef5b4 service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:23.331673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c391818-2e9c-4537-9180-7a54aaad5212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:23.332543 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c391818-2e9c-4537-9180-7a54aaad5212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:23.332826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c391818-2e9c-4537-9180-7a54aaad5212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:23.333145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c391818-2e9c-4537-9180-7a54aaad5212 admin admin] PATH_INFO: `/v3/users/1c1dedef8e3b49de94aeae690b7c8419` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:23.334414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c391818-2e9c-4537-9180-7a54aaad5212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c1dedef8e3b49de94aeae690b7c8419)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:23.336869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c391818-2e9c-4537-9180-7a54aaad5212 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:23.584641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5c391818-2e9c-4537-9180-7a54aaad5212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c1dedef8e3b49de94aeae690b7c8419'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:23.735100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5c391818-2e9c-4537-9180-7a54aaad5212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c1dedef8e3b49de94aeae690b7c8419'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:23.735761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5c391818-2e9c-4537-9180-7a54aaad5212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c1dedef8e3b49de94aeae690b7c8419'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:23.754609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5c391818-2e9c-4537-9180-7a54aaad5212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c1dedef8e3b49de94aeae690b7c8419'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:23.764023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5c391818-2e9c-4537-9180-7a54aaad5212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c1dedef8e3b49de94aeae690b7c8419'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:23.774423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 885/1768] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:23 2022] DELETE /identity/v3/users/1c1dedef8e3b49de94aeae690b7c8419 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:23.785727 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbffa23-6895-4e69-a1b7-2ce500e7cf5f None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:23.786307 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbffa23-6895-4e69-a1b7-2ce500e7cf5f None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:23.786484 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbffa23-6895-4e69-a1b7-2ce500e7cf5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:23.810512 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-8bbffa23-6895-4e69-a1b7-2ce500e7cf5f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:23.811942 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8bbffa23-6895-4e69-a1b7-2ce500e7cf5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:23.907524 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 884/1769] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:23.918966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c391818-2e9c-4537-9180-7a54aaad5212 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:23.921168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5c391818-2e9c-4537-9180-7a54aaad5212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:23.996126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c391818-2e9c-4537-9180-7a54aaad5212 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:24.000000 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419ddd8-18ec-4056-8ebb-2c8d2375031c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:24.000651 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419ddd8-18ec-4056-8ebb-2c8d2375031c service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:24.000969 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419ddd8-18ec-4056-8ebb-2c8d2375031c service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:24.001248 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419ddd8-18ec-4056-8ebb-2c8d2375031c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:24.011667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4419ddd8-18ec-4056-8ebb-2c8d2375031c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:24.013105 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4419ddd8-18ec-4056-8ebb-2c8d2375031c service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:24.023555 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 886/1770] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:24.276767 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:24.635550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbffa23-6895-4e69-a1b7-2ce500e7cf5f None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:24.648320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8bbffa23-6895-4e69-a1b7-2ce500e7cf5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:24.651929 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbffa23-6895-4e69-a1b7-2ce500e7cf5f None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:24.659943 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69c416-4771-40ef-b1fe-af953acb34b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:24.661852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69c416-4771-40ef-b1fe-af953acb34b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:24.662311 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69c416-4771-40ef-b1fe-af953acb34b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:24.662799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69c416-4771-40ef-b1fe-af953acb34b4 admin admin] PATH_INFO: `/v3/projects/a6c62bc1662845d2b66459d06df3d0eb` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:24.664685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69c416-4771-40ef-b1fe-af953acb34b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6c62bc1662845d2b66459d06df3d0eb)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:24.667572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69c416-4771-40ef-b1fe-af953acb34b4 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:24.918518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:24.953267 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ef69c416-4771-40ef-b1fe-af953acb34b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6c62bc1662845d2b66459d06df3d0eb', 'request_id': 'req-ef69c416-4771-40ef-b1fe-af953acb34b4'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:25.063602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ef69c416-4771-40ef-b1fe-af953acb34b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6c62bc1662845d2b66459d06df3d0eb', 'request_id': 'req-ef69c416-4771-40ef-b1fe-af953acb34b4'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:25.069538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ef69c416-4771-40ef-b1fe-af953acb34b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6c62bc1662845d2b66459d06df3d0eb', 'request_id': 'req-ef69c416-4771-40ef-b1fe-af953acb34b4'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:25.082417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ef69c416-4771-40ef-b1fe-af953acb34b4 admin admin] The token cache is being invalidate because project a6c62bc1662845d2b66459d06df3d0eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:25.082711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ef69c416-4771-40ef-b1fe-af953acb34b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:25.083380 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ef69c416-4771-40ef-b1fe-af953acb34b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:25.084989 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 885/1771] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:24 2022] DELETE /identity/v3/projects/a6c62bc1662845d2b66459d06df3d0eb => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:25.100260 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419ddd8-18ec-4056-8ebb-2c8d2375031c service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:25.113197 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419ddd8-18ec-4056-8ebb-2c8d2375031c service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:25.170742 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64f983-2b61-48cd-b6fa-d98428a11702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:25.171711 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64f983-2b61-48cd-b6fa-d98428a11702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:25.172136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64f983-2b61-48cd-b6fa-d98428a11702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:25.172480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64f983-2b61-48cd-b6fa-d98428a11702 admin admin] PATH_INFO: `/v3/users/cebfe156d4f54ebcac371ea13424b649` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:25.173772 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64f983-2b61-48cd-b6fa-d98428a11702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cebfe156d4f54ebcac371ea13424b649)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:25.175867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64f983-2b61-48cd-b6fa-d98428a11702 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:25.429689 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ee64f983-2b61-48cd-b6fa-d98428a11702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cebfe156d4f54ebcac371ea13424b649'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:25.575946 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ee64f983-2b61-48cd-b6fa-d98428a11702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cebfe156d4f54ebcac371ea13424b649'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:25.577030 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ee64f983-2b61-48cd-b6fa-d98428a11702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cebfe156d4f54ebcac371ea13424b649'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:25.595687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ee64f983-2b61-48cd-b6fa-d98428a11702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cebfe156d4f54ebcac371ea13424b649'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:25.612504 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ee64f983-2b61-48cd-b6fa-d98428a11702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cebfe156d4f54ebcac371ea13424b649'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:25.612504 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 887/1772] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:25 2022] DELETE /identity/v3/users/cebfe156d4f54ebcac371ea13424b649 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:26.199225 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69c416-4771-40ef-b1fe-af953acb34b4 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:26.201174 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ef69c416-4771-40ef-b1fe-af953acb34b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:26.223669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69c416-4771-40ef-b1fe-af953acb34b4 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:26.291429 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179c69f-08d6-439d-9712-b6259817b2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:26.292413 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179c69f-08d6-439d-9712-b6259817b2c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:26.292778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179c69f-08d6-439d-9712-b6259817b2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:26.293118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179c69f-08d6-439d-9712-b6259817b2c2 admin admin] PATH_INFO: `/v3/projects/bada7bf38324432682040e9c9cc04f30` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:26.294462 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4179c69f-08d6-439d-9712-b6259817b2c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bada7bf38324432682040e9c9cc04f30)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:26.296486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4179c69f-08d6-439d-9712-b6259817b2c2 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:26.410592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-4179c69f-08d6-439d-9712-b6259817b2c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bada7bf38324432682040e9c9cc04f30', 'request_id': 'req-4179c69f-08d6-439d-9712-b6259817b2c2'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:26.511950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-4179c69f-08d6-439d-9712-b6259817b2c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bada7bf38324432682040e9c9cc04f30', 'request_id': 'req-4179c69f-08d6-439d-9712-b6259817b2c2'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:26.518895 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-4179c69f-08d6-439d-9712-b6259817b2c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bada7bf38324432682040e9c9cc04f30', 'request_id': 'req-4179c69f-08d6-439d-9712-b6259817b2c2'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:26.529225 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-4179c69f-08d6-439d-9712-b6259817b2c2 admin admin] The token cache is being invalidate because project bada7bf38324432682040e9c9cc04f30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:26.529513 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-4179c69f-08d6-439d-9712-b6259817b2c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:26.530151 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-4179c69f-08d6-439d-9712-b6259817b2c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:26.531739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 886/1773] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:26 2022] DELETE /identity/v3/projects/bada7bf38324432682040e9c9cc04f30 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:26.537533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac458f-0292-4799-ae1f-d643af384105 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:26.538023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac458f-0292-4799-ae1f-d643af384105 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:26.538114 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac458f-0292-4799-ae1f-d643af384105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:26.557833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-74ac458f-0292-4799-ae1f-d643af384105 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:26.559027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-74ac458f-0292-4799-ae1f-d643af384105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:26.625505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 888/1774] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:26.634120 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03c9d9-d7a6-4972-83a2-9adc04361c32 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:26.634567 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03c9d9-d7a6-4972-83a2-9adc04361c32 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:26.634952 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03c9d9-d7a6-4972-83a2-9adc04361c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:26.654771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-0f03c9d9-d7a6-4972-83a2-9adc04361c32 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:26.655992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0f03c9d9-d7a6-4972-83a2-9adc04361c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:26.664306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 887/1775] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:26.669253 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac458f-0292-4799-ae1f-d643af384105 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:26.680279 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac458f-0292-4799-ae1f-d643af384105 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:26.686182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206cada-2674-4203-b2a4-87cbde65fe9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:26.686893 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206cada-2674-4203-b2a4-87cbde65fe9d admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:26.687185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206cada-2674-4203-b2a4-87cbde65fe9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:26.687471 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206cada-2674-4203-b2a4-87cbde65fe9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:26.688343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206cada-2674-4203-b2a4-87cbde65fe9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:32:26.688662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206cada-2674-4203-b2a4-87cbde65fe9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:26.690427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206cada-2674-4203-b2a4-87cbde65fe9d admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:26.704879 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 889/1776] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:32:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:32:26.709451 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03c9d9-d7a6-4972-83a2-9adc04361c32 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:26.721135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03c9d9-d7a6-4972-83a2-9adc04361c32 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:26.726608 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910e6e3-285d-471f-8569-282f61abf1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:26.727288 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910e6e3-285d-471f-8569-282f61abf1b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:26.727575 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910e6e3-285d-471f-8569-282f61abf1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:26.727857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910e6e3-285d-471f-8569-282f61abf1b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:26.729128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910e6e3-285d-471f-8569-282f61abf1b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:26.731100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910e6e3-285d-471f-8569-282f61abf1b2 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:26.799101 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:26.846593 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 888/1777] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:26 2022] POST /identity/v3/projects => generated 399 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:26.854284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206cada-2674-4203-b2a4-87cbde65fe9d admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:26.866114 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206cada-2674-4203-b2a4-87cbde65fe9d admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:26.926283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb28f6f-7498-44e1-997f-df5cc71b236f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:26.926960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb28f6f-7498-44e1-997f-df5cc71b236f admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:26.927298 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb28f6f-7498-44e1-997f-df5cc71b236f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:26.927642 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb28f6f-7498-44e1-997f-df5cc71b236f admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:26.928365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb28f6f-7498-44e1-997f-df5cc71b236f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:26.930375 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb28f6f-7498-44e1-997f-df5cc71b236f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:27.094581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 890/1778] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:26 2022] POST /identity/v3/users => generated 300 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:27.097324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910e6e3-285d-471f-8569-282f61abf1b2 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:27.109516 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910e6e3-285d-471f-8569-282f61abf1b2 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:27.116365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0e222-ee00-4e42-97a9-af8dfb4d7615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:27.116365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0e222-ee00-4e42-97a9-af8dfb4d7615 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:27.116535 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0e222-ee00-4e42-97a9-af8dfb4d7615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:27.116678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0e222-ee00-4e42-97a9-af8dfb4d7615 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:27.117769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0e222-ee00-4e42-97a9-af8dfb4d7615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:27.119786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0e222-ee00-4e42-97a9-af8dfb4d7615 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:27.133192 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 889/1779] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:27.138411 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb28f6f-7498-44e1-997f-df5cc71b236f admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:27.149868 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb28f6f-7498-44e1-997f-df5cc71b236f admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:27.160102 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8349c-2942-4107-b3d7-d81c936247a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:27.160786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8349c-2942-4107-b3d7-d81c936247a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:27.161068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8349c-2942-4107-b3d7-d81c936247a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:27.161345 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8349c-2942-4107-b3d7-d81c936247a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:27.162186 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8349c-2942-4107-b3d7-d81c936247a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:27.163869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8349c-2942-4107-b3d7-d81c936247a9 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:27.180299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 891/1780] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:27.181561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0e222-ee00-4e42-97a9-af8dfb4d7615 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:27.192055 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0e222-ee00-4e42-97a9-af8dfb4d7615 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:27.196569 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467abedc-7423-4aa7-b8ef-41090ba0bed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:27.197985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467abedc-7423-4aa7-b8ef-41090ba0bed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:27.198294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467abedc-7423-4aa7-b8ef-41090ba0bed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:27.198572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467abedc-7423-4aa7-b8ef-41090ba0bed6 admin admin] PATH_INFO: `/v3/projects/76b557c9d2ba4727889852cc860db0d7/users/68cdb459381643c4923a61e0689c4d2d/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:27.217826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467abedc-7423-4aa7-b8ef-41090ba0bed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76b557c9d2ba4727889852cc860db0d7, user_id=68cdb459381643c4923a61e0689c4d2d, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:27.220523 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467abedc-7423-4aa7-b8ef-41090ba0bed6 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:27.252428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:27.291270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 890/1781] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:27 2022] PUT /identity/v3/projects/76b557c9d2ba4727889852cc860db0d7/users/68cdb459381643c4923a61e0689c4d2d/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:27.302349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce0fab-ccdf-40b5-884a-cc484d7af8ac None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:27.302715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce0fab-ccdf-40b5-884a-cc484d7af8ac None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:27.302990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce0fab-ccdf-40b5-884a-cc484d7af8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:27.337017 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-10ce0fab-ccdf-40b5-884a-cc484d7af8ac None None] MFA Rules not processed for user `68cdb459381643c4923a61e0689c4d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:27.338890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-10ce0fab-ccdf-40b5-884a-cc484d7af8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:27.419756 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 892/1782] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:27.425760 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467abedc-7423-4aa7-b8ef-41090ba0bed6 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:27.529164 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467abedc-7423-4aa7-b8ef-41090ba0bed6 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:27.537550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a43dc-689b-4a66-8aa6-3db01c09d5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:27.538744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a43dc-689b-4a66-8aa6-3db01c09d5c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:27.539383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a43dc-689b-4a66-8aa6-3db01c09d5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:27.539723 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a43dc-689b-4a66-8aa6-3db01c09d5c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:27.544051 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a43dc-689b-4a66-8aa6-3db01c09d5c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:27.544258 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a43dc-689b-4a66-8aa6-3db01c09d5c2 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:27.756248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 891/1783] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:27 2022] POST /identity/v3/projects => generated 401 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:27.761734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce0fab-ccdf-40b5-884a-cc484d7af8ac None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:27.774296 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce0fab-ccdf-40b5-884a-cc484d7af8ac None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:27.839226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a5591-2aee-4c89-8e07-e07a5fe0fc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:27.839845 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a5591-2aee-4c89-8e07-e07a5fe0fc5e admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:27.840180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a5591-2aee-4c89-8e07-e07a5fe0fc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:27.840508 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a5591-2aee-4c89-8e07-e07a5fe0fc5e admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:27.841464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a5591-2aee-4c89-8e07-e07a5fe0fc5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:27.844024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a5591-2aee-4c89-8e07-e07a5fe0fc5e admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:27.953076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 893/1784] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:27 2022] POST /identity/v3/users => generated 307 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:27.958402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a43dc-689b-4a66-8aa6-3db01c09d5c2 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:27.970432 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a43dc-689b-4a66-8aa6-3db01c09d5c2 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:27.975868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85057970-f8bb-414b-b365-7577e6e4355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:27.976547 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85057970-f8bb-414b-b365-7577e6e4355f admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:27.976827 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85057970-f8bb-414b-b365-7577e6e4355f admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:27.977107 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85057970-f8bb-414b-b365-7577e6e4355f admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:27.977840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85057970-f8bb-414b-b365-7577e6e4355f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:27.980024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85057970-f8bb-414b-b365-7577e6e4355f admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:27.993086 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 892/1785] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:27.999584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a5591-2aee-4c89-8e07-e07a5fe0fc5e admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:28.012588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a5591-2aee-4c89-8e07-e07a5fe0fc5e admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:28.017538 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed7bac-8ec3-404e-8ff2-cd8b4e336572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:28.018554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed7bac-8ec3-404e-8ff2-cd8b4e336572 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:28.018914 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed7bac-8ec3-404e-8ff2-cd8b4e336572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:28.019252 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed7bac-8ec3-404e-8ff2-cd8b4e336572 admin admin] PATH_INFO: `/v3/projects/09f0a4400b414d729e1eeb3714cf5bd5/users/fc084ce1cb384bf8b01336698d13221d/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:28.037156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed7bac-8ec3-404e-8ff2-cd8b4e336572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09f0a4400b414d729e1eeb3714cf5bd5, user_id=fc084ce1cb384bf8b01336698d13221d, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:28.040322 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed7bac-8ec3-404e-8ff2-cd8b4e336572 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:28.208786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 894/1786] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:27 2022] PUT /identity/v3/projects/09f0a4400b414d729e1eeb3714cf5bd5/users/fc084ce1cb384bf8b01336698d13221d/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:28.215205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85057970-f8bb-414b-b365-7577e6e4355f admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:28.232771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85057970-f8bb-414b-b365-7577e6e4355f admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:28.326748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b32a03-de16-4960-81fd-8346317646b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:28.330060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b32a03-de16-4960-81fd-8346317646b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:28.330060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b32a03-de16-4960-81fd-8346317646b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:28.330060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b32a03-de16-4960-81fd-8346317646b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:28.330060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b32a03-de16-4960-81fd-8346317646b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:28.331125 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b32a03-de16-4960-81fd-8346317646b5 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:28.347065 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 893/1787] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:28.348651 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed7bac-8ec3-404e-8ff2-cd8b4e336572 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:28.361752 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed7bac-8ec3-404e-8ff2-cd8b4e336572 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:28.367090 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77af5a-fec0-401a-9d1f-241d150d757e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:28.369559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77af5a-fec0-401a-9d1f-241d150d757e admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:28.369955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77af5a-fec0-401a-9d1f-241d150d757e admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:28.370432 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77af5a-fec0-401a-9d1f-241d150d757e admin admin] PATH_INFO: `/v3/projects/09f0a4400b414d729e1eeb3714cf5bd5/users/fc084ce1cb384bf8b01336698d13221d/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:28.372880 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77af5a-fec0-401a-9d1f-241d150d757e admin admin] RBAC: Authorizing `identity:create_grant(project_id=09f0a4400b414d729e1eeb3714cf5bd5, user_id=fc084ce1cb384bf8b01336698d13221d, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:28.377715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77af5a-fec0-401a-9d1f-241d150d757e admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:28.396523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 895/1788] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:28 2022] PUT /identity/v3/projects/09f0a4400b414d729e1eeb3714cf5bd5/users/fc084ce1cb384bf8b01336698d13221d/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:28.412538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377f38e-1df6-4de5-9ed6-74cbc05afa47 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:28.412832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377f38e-1df6-4de5-9ed6-74cbc05afa47 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:28.413117 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377f38e-1df6-4de5-9ed6-74cbc05afa47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:28.450321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-1377f38e-1df6-4de5-9ed6-74cbc05afa47 None None] MFA Rules not processed for user `fc084ce1cb384bf8b01336698d13221d`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:28.451814 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1377f38e-1df6-4de5-9ed6-74cbc05afa47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:28.538464 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 894/1789] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:28 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:28.550834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77af5a-fec0-401a-9d1f-241d150d757e admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:28.553045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ca77af5a-fec0-401a-9d1f-241d150d757e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:28.626937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77af5a-fec0-401a-9d1f-241d150d757e admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:28.630708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c0717-d077-457c-8b94-cdf4a854835c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:28.631641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c0717-d077-457c-8b94-cdf4a854835c service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:28.631825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c0717-d077-457c-8b94-cdf4a854835c service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:28.632106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c0717-d077-457c-8b94-cdf4a854835c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:28.642822 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c0717-d077-457c-8b94-cdf4a854835c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:28.644253 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c0717-d077-457c-8b94-cdf4a854835c service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:28.655266 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 896/1790] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:28 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:28.921191 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377f38e-1df6-4de5-9ed6-74cbc05afa47 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:28.930806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377f38e-1df6-4de5-9ed6-74cbc05afa47 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:28.934542 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea37560-e476-405b-8f85-b7b1c19b6b91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:28.935206 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea37560-e476-405b-8f85-b7b1c19b6b91 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:28.935483 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea37560-e476-405b-8f85-b7b1c19b6b91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:28.935754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea37560-e476-405b-8f85-b7b1c19b6b91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:28.994688 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea37560-e476-405b-8f85-b7b1c19b6b91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:28.996169 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea37560-e476-405b-8f85-b7b1c19b6b91 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:29.006673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 895/1791] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:29.096543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:30.387033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:31.719739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:32.533482 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c0717-d077-457c-8b94-cdf4a854835c service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:32.538044 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-af6c0717-d077-457c-8b94-cdf4a854835c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:32.661277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c0717-d077-457c-8b94-cdf4a854835c service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:32.665908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7820b8e-c6b5-4d43-b5c3-df8ff684341a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:32.666862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7820b8e-c6b5-4d43-b5c3-df8ff684341a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:32.667110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7820b8e-c6b5-4d43-b5c3-df8ff684341a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:32.667371 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7820b8e-c6b5-4d43-b5c3-df8ff684341a admin admin] PATH_INFO: `/v3/users/d05979493394438dadc186c47c590457` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:32.668602 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7820b8e-c6b5-4d43-b5c3-df8ff684341a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d05979493394438dadc186c47c590457)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:32.670602 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7820b8e-c6b5-4d43-b5c3-df8ff684341a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:32.945314 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:33.011259 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-b7820b8e-c6b5-4d43-b5c3-df8ff684341a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd05979493394438dadc186c47c590457'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:33.186530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-b7820b8e-c6b5-4d43-b5c3-df8ff684341a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd05979493394438dadc186c47c590457'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:33.187160 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-b7820b8e-c6b5-4d43-b5c3-df8ff684341a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd05979493394438dadc186c47c590457'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:33.208490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-b7820b8e-c6b5-4d43-b5c3-df8ff684341a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd05979493394438dadc186c47c590457'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:33.224706 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-b7820b8e-c6b5-4d43-b5c3-df8ff684341a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd05979493394438dadc186c47c590457'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:33.242764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 897/1792] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:32 2022] DELETE /identity/v3/users/d05979493394438dadc186c47c590457 => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:33.248099 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a9931-a8c5-488c-840d-935a4d9928f9 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:33.249960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a9931-a8c5-488c-840d-935a4d9928f9 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:33.251308 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a9931-a8c5-488c-840d-935a4d9928f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:33.296842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-b87a9931-a8c5-488c-840d-935a4d9928f9 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:33.298479 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b87a9931-a8c5-488c-840d-935a4d9928f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:33.450127 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 896/1793] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:32:33.463965 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7820b8e-c6b5-4d43-b5c3-df8ff684341a admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:33.472069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b7820b8e-c6b5-4d43-b5c3-df8ff684341a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:33.600932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7820b8e-c6b5-4d43-b5c3-df8ff684341a admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:33.610870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ad391-8b47-4e5f-bbe8-b91f5e50cd41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:33.611544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ad391-8b47-4e5f-bbe8-b91f5e50cd41 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:33.611829 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ad391-8b47-4e5f-bbe8-b91f5e50cd41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:33.612105 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ad391-8b47-4e5f-bbe8-b91f5e50cd41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:33.631294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ad391-8b47-4e5f-bbe8-b91f5e50cd41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:33.633156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ad391-8b47-4e5f-bbe8-b91f5e50cd41 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:33.651935 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 898/1794] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:34.363727 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a9931-a8c5-488c-840d-935a4d9928f9 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:34.366335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b87a9931-a8c5-488c-840d-935a4d9928f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:34.407880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a9931-a8c5-488c-840d-935a4d9928f9 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:34.413894 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:34.418727 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:34.419204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:34.423538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5 admin admin] PATH_INFO: `/v3/projects/0df3a7f0776c411db4604ac7bfe5719a` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:34.423538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0df3a7f0776c411db4604ac7bfe5719a)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:34.423538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:34.594996 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0df3a7f0776c411db4604ac7bfe5719a', 'request_id': 'req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:34.682098 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0df3a7f0776c411db4604ac7bfe5719a', 'request_id': 'req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:34.690922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0df3a7f0776c411db4604ac7bfe5719a', 'request_id': 'req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:34.708856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5 admin admin] The token cache is being invalidate because project 0df3a7f0776c411db4604ac7bfe5719a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:34.709361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:34.711505 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:34.718807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 897/1795] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:34 2022] DELETE /identity/v3/projects/0df3a7f0776c411db4604ac7bfe5719a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:34.721651 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ad391-8b47-4e5f-bbe8-b91f5e50cd41 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:34.733832 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ad391-8b47-4e5f-bbe8-b91f5e50cd41 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:34.814165 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab2faf-d32a-49c0-86f3-b7ffc7bffd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:34.816695 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab2faf-d32a-49c0-86f3-b7ffc7bffd89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:34.817531 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab2faf-d32a-49c0-86f3-b7ffc7bffd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:34.818157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab2faf-d32a-49c0-86f3-b7ffc7bffd89 admin admin] PATH_INFO: `/v3/users/733c17f3f48f45f697a70a2af9b7c902` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:34.820548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab2faf-d32a-49c0-86f3-b7ffc7bffd89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=733c17f3f48f45f697a70a2af9b7c902)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:34.825377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab2faf-d32a-49c0-86f3-b7ffc7bffd89 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:35.080470 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-75ab2faf-d32a-49c0-86f3-b7ffc7bffd89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '733c17f3f48f45f697a70a2af9b7c902'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:35.116284 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:35.201614 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-75ab2faf-d32a-49c0-86f3-b7ffc7bffd89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '733c17f3f48f45f697a70a2af9b7c902'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:35.202598 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-75ab2faf-d32a-49c0-86f3-b7ffc7bffd89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '733c17f3f48f45f697a70a2af9b7c902'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:35.218919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-75ab2faf-d32a-49c0-86f3-b7ffc7bffd89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '733c17f3f48f45f697a70a2af9b7c902'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:35.230736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-75ab2faf-d32a-49c0-86f3-b7ffc7bffd89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '733c17f3f48f45f697a70a2af9b7c902'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:35.250002 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 899/1796] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:34 2022] DELETE /identity/v3/users/733c17f3f48f45f697a70a2af9b7c902 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:35.346937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:35.445013 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:35.446392 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:35.462411 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4d09a-480b-4189-b15b-f2984f7e3bc5 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:35.586006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429db133-c700-4931-a613-b43976d2d220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:35.587064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429db133-c700-4931-a613-b43976d2d220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:35.587523 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429db133-c700-4931-a613-b43976d2d220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:35.588307 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429db133-c700-4931-a613-b43976d2d220 admin admin] PATH_INFO: `/v3/users/68cdb459381643c4923a61e0689c4d2d` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:35.591070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429db133-c700-4931-a613-b43976d2d220 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68cdb459381643c4923a61e0689c4d2d)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:35.594519 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429db133-c700-4931-a613-b43976d2d220 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:35.818571 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-429db133-c700-4931-a613-b43976d2d220 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68cdb459381643c4923a61e0689c4d2d'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:35.871507 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab2faf-d32a-49c0-86f3-b7ffc7bffd89 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:35.875524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-75ab2faf-d32a-49c0-86f3-b7ffc7bffd89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:35.889026 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-429db133-c700-4931-a613-b43976d2d220 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68cdb459381643c4923a61e0689c4d2d'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:35.902232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-429db133-c700-4931-a613-b43976d2d220 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68cdb459381643c4923a61e0689c4d2d'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:35.903356 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-429db133-c700-4931-a613-b43976d2d220 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68cdb459381643c4923a61e0689c4d2d'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:35.926887 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab2faf-d32a-49c0-86f3-b7ffc7bffd89 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:35.932526 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-429db133-c700-4931-a613-b43976d2d220 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68cdb459381643c4923a61e0689c4d2d'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:35.940510 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edd68f-f773-42c8-9c69-aace711c140b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:35.943874 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edd68f-f773-42c8-9c69-aace711c140b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:35.944771 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edd68f-f773-42c8-9c69-aace711c140b admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:35.946410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edd68f-f773-42c8-9c69-aace711c140b admin admin] PATH_INFO: `/v3/projects/a3651eaccb084707aeedcad163e7953b` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:35.947962 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edd68f-f773-42c8-9c69-aace711c140b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3651eaccb084707aeedcad163e7953b)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:35.950448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edd68f-f773-42c8-9c69-aace711c140b admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:35.952779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 898/1797] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:35 2022] DELETE /identity/v3/users/68cdb459381643c4923a61e0689c4d2d => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:35.961745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7b7f1-9edd-4df4-a4c3-cd60eef1e1b8 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:35.963221 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7b7f1-9edd-4df4-a4c3-cd60eef1e1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:35.964543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7b7f1-9edd-4df4-a4c3-cd60eef1e1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:36.018883 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-f9a7b7f1-9edd-4df4-a4c3-cd60eef1e1b8 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:36.020224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f9a7b7f1-9edd-4df4-a4c3-cd60eef1e1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:36.158414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 899/1798] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:32:36.163005 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-46edd68f-f773-42c8-9c69-aace711c140b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3651eaccb084707aeedcad163e7953b', 'request_id': 'req-46edd68f-f773-42c8-9c69-aace711c140b'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:36.178056 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7b7f1-9edd-4df4-a4c3-cd60eef1e1b8 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:36.179727 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f9a7b7f1-9edd-4df4-a4c3-cd60eef1e1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:36.251054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7b7f1-9edd-4df4-a4c3-cd60eef1e1b8 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:36.265573 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-46edd68f-f773-42c8-9c69-aace711c140b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3651eaccb084707aeedcad163e7953b', 'request_id': 'req-46edd68f-f773-42c8-9c69-aace711c140b'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:36.274632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-46edd68f-f773-42c8-9c69-aace711c140b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3651eaccb084707aeedcad163e7953b', 'request_id': 'req-46edd68f-f773-42c8-9c69-aace711c140b'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:36.287353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-46edd68f-f773-42c8-9c69-aace711c140b admin admin] The token cache is being invalidate because project a3651eaccb084707aeedcad163e7953b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:36.287635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-46edd68f-f773-42c8-9c69-aace711c140b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:36.288533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-46edd68f-f773-42c8-9c69-aace711c140b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:36.290237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 900/1799] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:35 2022] DELETE /identity/v3/projects/a3651eaccb084707aeedcad163e7953b => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:36.295428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf527c0-7438-4d53-a672-14b394cfd3db None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:36.295988 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf527c0-7438-4d53-a672-14b394cfd3db None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:36.296275 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf527c0-7438-4d53-a672-14b394cfd3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:36.317177 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-5bf527c0-7438-4d53-a672-14b394cfd3db None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:36.319245 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5bf527c0-7438-4d53-a672-14b394cfd3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:36.331787 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d82a8e-ed54-4d91-916e-719fbdda047f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:36.332357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d82a8e-ed54-4d91-916e-719fbdda047f service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:36.333387 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d82a8e-ed54-4d91-916e-719fbdda047f service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:36.336935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d82a8e-ed54-4d91-916e-719fbdda047f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:36.336935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-87d82a8e-ed54-4d91-916e-719fbdda047f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:36.402398 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 901/1800] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:36.406715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754b89b-7968-49d8-af04-7e70a10da6ab None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:36.407231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754b89b-7968-49d8-af04-7e70a10da6ab None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:36.407641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754b89b-7968-49d8-af04-7e70a10da6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:36.428822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d82a8e-ed54-4d91-916e-719fbdda047f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:36.429152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-6754b89b-7968-49d8-af04-7e70a10da6ab None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:36.430815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-6754b89b-7968-49d8-af04-7e70a10da6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:36.431496 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d82a8e-ed54-4d91-916e-719fbdda047f service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:36.440997 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 902/1801] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:32:36.447808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754b89b-7968-49d8-af04-7e70a10da6ab None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:36.449099 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 900/1802] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:32:36.458350 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754b89b-7968-49d8-af04-7e70a10da6ab None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:36.464517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2534ba3-ff9b-47c7-9599-a6dd79e6ae13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:36.465279 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2534ba3-ff9b-47c7-9599-a6dd79e6ae13 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:36.465632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2534ba3-ff9b-47c7-9599-a6dd79e6ae13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:36.466542 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2534ba3-ff9b-47c7-9599-a6dd79e6ae13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:36.467274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2534ba3-ff9b-47c7-9599-a6dd79e6ae13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:32:36.468325 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2534ba3-ff9b-47c7-9599-a6dd79e6ae13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:36.469500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2534ba3-ff9b-47c7-9599-a6dd79e6ae13 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:36.486837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 903/1803] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:32:36.491516 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d82a8e-ed54-4d91-916e-719fbdda047f service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:36.503474 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d82a8e-ed54-4d91-916e-719fbdda047f service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:36.509852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346bd40-da87-49d2-886d-60615c2f460a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:36.510711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346bd40-da87-49d2-886d-60615c2f460a admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:36.511542 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346bd40-da87-49d2-886d-60615c2f460a admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:36.511812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346bd40-da87-49d2-886d-60615c2f460a admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:36.512675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346bd40-da87-49d2-886d-60615c2f460a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:36.514841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346bd40-da87-49d2-886d-60615c2f460a admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:36.761452 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 901/1804] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:36 2022] POST /identity/v3/projects => generated 387 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:32:36.762828 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2534ba3-ff9b-47c7-9599-a6dd79e6ae13 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:36.774951 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2534ba3-ff9b-47c7-9599-a6dd79e6ae13 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:36.782051 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e34826-7f0a-4bbf-b4ff-3ea7bd09805b None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:36.782406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e34826-7f0a-4bbf-b4ff-3ea7bd09805b None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:36.782641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e34826-7f0a-4bbf-b4ff-3ea7bd09805b None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:36.784119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 902/1805] 10.222.0.78 () {54 vars in 921 bytes} [Wed Jul 20 15:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:32:36.796480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4361a9-bd40-4dc8-b6ff-f49a41840e9e None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:36.796766 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4361a9-bd40-4dc8-b6ff-f49a41840e9e None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:36.797218 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4361a9-bd40-4dc8-b6ff-f49a41840e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:36.828274 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-fe4361a9-bd40-4dc8-b6ff-f49a41840e9e None None] MFA Rules not processed for user `a4a24d5a375945779fa626fae25ac237`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:36.830192 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-fe4361a9-bd40-4dc8-b6ff-f49a41840e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:36.869759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4580c-ad7c-4061-bcdb-78fc1b3537fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:36.870513 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4580c-ad7c-4061-bcdb-78fc1b3537fc admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:36.871052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4580c-ad7c-4061-bcdb-78fc1b3537fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:36.871447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4580c-ad7c-4061-bcdb-78fc1b3537fc admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:36.873122 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4580c-ad7c-4061-bcdb-78fc1b3537fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:36.876640 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4580c-ad7c-4061-bcdb-78fc1b3537fc admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:36.909421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:36.984493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 903/1806] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Jul 20 15:32:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:32:37.007442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 904/1807] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:36 2022] POST /identity/v3/users => generated 294 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:37.009245 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4361a9-bd40-4dc8-b6ff-f49a41840e9e None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:37.033510 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4361a9-bd40-4dc8-b6ff-f49a41840e9e None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:37.053663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fba548-d930-4749-acea-9bec91f755ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:37.053663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fba548-d930-4749-acea-9bec91f755ef admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:37.053663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fba548-d930-4749-acea-9bec91f755ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:37.053663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fba548-d930-4749-acea-9bec91f755ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:37.054079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fba548-d930-4749-acea-9bec91f755ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:37.055312 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fba548-d930-4749-acea-9bec91f755ef admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:37.086448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 904/1808] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:37.092694 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4580c-ad7c-4061-bcdb-78fc1b3537fc admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:37.122180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4580c-ad7c-4061-bcdb-78fc1b3537fc admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:37.129576 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22dde1-bd0e-4eca-85c0-875509233bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:37.130365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22dde1-bd0e-4eca-85c0-875509233bbe admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:37.130676 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22dde1-bd0e-4eca-85c0-875509233bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:37.130957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22dde1-bd0e-4eca-85c0-875509233bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:37.131729 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22dde1-bd0e-4eca-85c0-875509233bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:37.133501 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22dde1-bd0e-4eca-85c0-875509233bbe admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:37.146033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 905/1809] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:37.150881 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fba548-d930-4749-acea-9bec91f755ef admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:37.164801 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fba548-d930-4749-acea-9bec91f755ef admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:37.174803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a1754-f17f-4ba4-a779-f8c9741d29b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:37.175967 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a1754-f17f-4ba4-a779-f8c9741d29b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:37.177114 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a1754-f17f-4ba4-a779-f8c9741d29b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:37.178678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a1754-f17f-4ba4-a779-f8c9741d29b1 admin admin] PATH_INFO: `/v3/projects/73192402244d44739ff3bf04bdd9269b/users/37fee75a0acc4c3f8ff81793c3fb7b39/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:37.206626 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22dde1-bd0e-4eca-85c0-875509233bbe admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:37.208965 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-2d22dde1-bd0e-4eca-85c0-875509233bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:37.236261 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a1754-f17f-4ba4-a779-f8c9741d29b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73192402244d44739ff3bf04bdd9269b, user_id=37fee75a0acc4c3f8ff81793c3fb7b39, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:37.242518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22dde1-bd0e-4eca-85c0-875509233bbe admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:37.245404 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a1754-f17f-4ba4-a779-f8c9741d29b1 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:37.250469 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee02d98-8a50-4b19-9cb3-aee45999e232 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:37.251358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee02d98-8a50-4b19-9cb3-aee45999e232 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:37.251818 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee02d98-8a50-4b19-9cb3-aee45999e232 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:37.252321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee02d98-8a50-4b19-9cb3-aee45999e232 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:37.265290 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee02d98-8a50-4b19-9cb3-aee45999e232 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:37.269265 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee02d98-8a50-4b19-9cb3-aee45999e232 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:37.302530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 906/1810] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:37.386383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 905/1811] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:37 2022] PUT /identity/v3/projects/73192402244d44739ff3bf04bdd9269b/users/37fee75a0acc4c3f8ff81793c3fb7b39/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:37.397934 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2f9c1-8d40-4c7f-bf73-f9941460da8e None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:37.398537 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2f9c1-8d40-4c7f-bf73-f9941460da8e None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:37.398964 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2f9c1-8d40-4c7f-bf73-f9941460da8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:37.432721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-44f2f9c1-8d40-4c7f-bf73-f9941460da8e None None] MFA Rules not processed for user `37fee75a0acc4c3f8ff81793c3fb7b39`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:37.434859 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-44f2f9c1-8d40-4c7f-bf73-f9941460da8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:37.455560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a1754-f17f-4ba4-a779-f8c9741d29b1 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:37.459108 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8b2a1754-f17f-4ba4-a779-f8c9741d29b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:37.561136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 907/1812] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:37.563006 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2f9c1-8d40-4c7f-bf73-f9941460da8e None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:37.568397 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a1754-f17f-4ba4-a779-f8c9741d29b1 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:37.577324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:37.577324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:37.577324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:37.577324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25 admin admin] PATH_INFO: `/v3/projects/76b557c9d2ba4727889852cc860db0d7` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:37.579052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76b557c9d2ba4727889852cc860db0d7)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:37.581215 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:37.582521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2f9c1-8d40-4c7f-bf73-f9941460da8e None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:37.589345 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d79e5-ff57-48be-8af4-6a3f75128da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:37.590162 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d79e5-ff57-48be-8af4-6a3f75128da3 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:37.591221 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d79e5-ff57-48be-8af4-6a3f75128da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:37.591921 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d79e5-ff57-48be-8af4-6a3f75128da3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:37.593069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d79e5-ff57-48be-8af4-6a3f75128da3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:37.595791 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d79e5-ff57-48be-8af4-6a3f75128da3 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:37.696217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 908/1813] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:37 2022] POST /identity/v3/projects => generated 389 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:37.700609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d79e5-ff57-48be-8af4-6a3f75128da3 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:37.755714 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76b557c9d2ba4727889852cc860db0d7', 'request_id': 'req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:37.769917 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d79e5-ff57-48be-8af4-6a3f75128da3 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:37.834948 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c17bb-27fe-4442-b8df-432ef2ca62cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:37.835937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c17bb-27fe-4442-b8df-432ef2ca62cb admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:37.836939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c17bb-27fe-4442-b8df-432ef2ca62cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:37.837306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c17bb-27fe-4442-b8df-432ef2ca62cb admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:37.838059 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c17bb-27fe-4442-b8df-432ef2ca62cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:37.841063 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c17bb-27fe-4442-b8df-432ef2ca62cb admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:37.861454 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76b557c9d2ba4727889852cc860db0d7', 'request_id': 'req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:37.868362 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76b557c9d2ba4727889852cc860db0d7', 'request_id': 'req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:37.879604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25 admin admin] The token cache is being invalidate because project 76b557c9d2ba4727889852cc860db0d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:37.879895 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:37.880587 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:37.882167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 906/1814] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:37 2022] DELETE /identity/v3/projects/76b557c9d2ba4727889852cc860db0d7 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:37.886477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:37.887896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:37.909401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8ac49-be97-4e23-bdbe-d4674aaf9f25 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:37.916212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcbf1d-be8c-44ed-8e3f-ab99802d3b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:37.917114 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcbf1d-be8c-44ed-8e3f-ab99802d3b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:37.917420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcbf1d-be8c-44ed-8e3f-ab99802d3b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:37.917717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcbf1d-be8c-44ed-8e3f-ab99802d3b12 admin admin] PATH_INFO: `/v3/users/fc084ce1cb384bf8b01336698d13221d` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:37.919212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bcbf1d-be8c-44ed-8e3f-ab99802d3b12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc084ce1cb384bf8b01336698d13221d)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:37.921362 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bcbf1d-be8c-44ed-8e3f-ab99802d3b12 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:37.974102 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 909/1815] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:37 2022] POST /identity/v3/users => generated 301 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:32:37.978905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c17bb-27fe-4442-b8df-432ef2ca62cb admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:37.981490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-2e0c17bb-27fe-4442-b8df-432ef2ca62cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:37.994796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c17bb-27fe-4442-b8df-432ef2ca62cb admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:38.001974 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf75bea-af67-4813-80d3-2ee899c41eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:38.006490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf75bea-af67-4813-80d3-2ee899c41eaf admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:38.006490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf75bea-af67-4813-80d3-2ee899c41eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:38.006490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf75bea-af67-4813-80d3-2ee899c41eaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:38.006490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf75bea-af67-4813-80d3-2ee899c41eaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:38.006490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf75bea-af67-4813-80d3-2ee899c41eaf admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:38.022554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 910/1816] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:32:38.025525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf75bea-af67-4813-80d3-2ee899c41eaf admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:38.048803 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf75bea-af67-4813-80d3-2ee899c41eaf admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:38.059027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b4a343-571d-42ed-8545-ecd8ce609887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:38.060115 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4a343-571d-42ed-8545-ecd8ce609887 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:38.060544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4a343-571d-42ed-8545-ecd8ce609887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:38.060964 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4a343-571d-42ed-8545-ecd8ce609887 admin admin] PATH_INFO: `/v3/projects/297ad5303700438686e655c6df1fe40a/users/f537bcffc65b467ba11c9b758cdda68e/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:38.079136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b4a343-571d-42ed-8545-ecd8ce609887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=297ad5303700438686e655c6df1fe40a, user_id=f537bcffc65b467ba11c9b758cdda68e, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:38.082076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b4a343-571d-42ed-8545-ecd8ce609887 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:38.123688 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:38.198817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-45bcbf1d-be8c-44ed-8e3f-ab99802d3b12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc084ce1cb384bf8b01336698d13221d'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:38.249563 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 911/1817] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:38 2022] PUT /identity/v3/projects/297ad5303700438686e655c6df1fe40a/users/f537bcffc65b467ba11c9b758cdda68e/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:38.250845 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b4a343-571d-42ed-8545-ecd8ce609887 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:38.252684 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-18b4a343-571d-42ed-8545-ecd8ce609887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:38.294487 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-45bcbf1d-be8c-44ed-8e3f-ab99802d3b12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc084ce1cb384bf8b01336698d13221d'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:38.304950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-45bcbf1d-be8c-44ed-8e3f-ab99802d3b12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc084ce1cb384bf8b01336698d13221d'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:38.305782 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-45bcbf1d-be8c-44ed-8e3f-ab99802d3b12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc084ce1cb384bf8b01336698d13221d'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:38.344263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-45bcbf1d-be8c-44ed-8e3f-ab99802d3b12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc084ce1cb384bf8b01336698d13221d'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:38.363243 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b4a343-571d-42ed-8545-ecd8ce609887 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:38.370202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d83ac0-3d5c-4e3f-ab6a-cfd2fdb5723d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:38.371386 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d83ac0-3d5c-4e3f-ab6a-cfd2fdb5723d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:38.371909 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d83ac0-3d5c-4e3f-ab6a-cfd2fdb5723d admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:38.373071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d83ac0-3d5c-4e3f-ab6a-cfd2fdb5723d admin admin] PATH_INFO: `/v3/users/e950c6b6bbb34b9697e3a13585b29b7c` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:38.373955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 907/1818] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:37 2022] DELETE /identity/v3/users/fc084ce1cb384bf8b01336698d13221d => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:38.374297 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d83ac0-3d5c-4e3f-ab6a-cfd2fdb5723d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e950c6b6bbb34b9697e3a13585b29b7c)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:38.375449 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcbf1d-be8c-44ed-8e3f-ab99802d3b12 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:38.377308 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d83ac0-3d5c-4e3f-ab6a-cfd2fdb5723d admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:38.379433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-45bcbf1d-be8c-44ed-8e3f-ab99802d3b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:38.528198 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcbf1d-be8c-44ed-8e3f-ab99802d3b12 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:38.546832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b409a46-699e-410f-b308-0f251ba836e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:38.547967 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b409a46-699e-410f-b308-0f251ba836e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:38.548761 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b409a46-699e-410f-b308-0f251ba836e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:38.549247 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b409a46-699e-410f-b308-0f251ba836e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:38.550422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b409a46-699e-410f-b308-0f251ba836e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:38.553153 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b409a46-699e-410f-b308-0f251ba836e0 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:38.577623 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 908/1819] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:38.586069 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b409a46-699e-410f-b308-0f251ba836e0 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:38.617522 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-07d83ac0-3d5c-4e3f-ab6a-cfd2fdb5723d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e950c6b6bbb34b9697e3a13585b29b7c'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:38.621930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b409a46-699e-410f-b308-0f251ba836e0 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:38.630672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81436361-9794-4829-81c3-21acb118e8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:38.632438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81436361-9794-4829-81c3-21acb118e8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:38.632438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81436361-9794-4829-81c3-21acb118e8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:38.632438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81436361-9794-4829-81c3-21acb118e8b2 admin admin] PATH_INFO: `/v3/projects/297ad5303700438686e655c6df1fe40a/users/f537bcffc65b467ba11c9b758cdda68e/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:38.638206 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81436361-9794-4829-81c3-21acb118e8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=297ad5303700438686e655c6df1fe40a, user_id=f537bcffc65b467ba11c9b758cdda68e, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:38.641278 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81436361-9794-4829-81c3-21acb118e8b2 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:38.678494 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 909/1820] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:38 2022] PUT /identity/v3/projects/297ad5303700438686e655c6df1fe40a/users/f537bcffc65b467ba11c9b758cdda68e/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:38.689861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac34151-828a-4851-a930-6fda630db265 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:38.690182 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac34151-828a-4851-a930-6fda630db265 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:38.692654 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac34151-828a-4851-a930-6fda630db265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:38.725049 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-07d83ac0-3d5c-4e3f-ab6a-cfd2fdb5723d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e950c6b6bbb34b9697e3a13585b29b7c'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:38.726674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-07d83ac0-3d5c-4e3f-ab6a-cfd2fdb5723d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e950c6b6bbb34b9697e3a13585b29b7c'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:38.751443 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-cac34151-828a-4851-a930-6fda630db265 None None] MFA Rules not processed for user `f537bcffc65b467ba11c9b758cdda68e`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:38.757411 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-07d83ac0-3d5c-4e3f-ab6a-cfd2fdb5723d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e950c6b6bbb34b9697e3a13585b29b7c'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:38.758010 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-cac34151-828a-4851-a930-6fda630db265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:38.793084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-07d83ac0-3d5c-4e3f-ab6a-cfd2fdb5723d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e950c6b6bbb34b9697e3a13585b29b7c'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:38.811827 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 912/1821] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:38 2022] DELETE /identity/v3/users/e950c6b6bbb34b9697e3a13585b29b7c => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:38.815801 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39855ab6-42f3-477d-8f8b-31fc415c22a3 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:38.815801 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39855ab6-42f3-477d-8f8b-31fc415c22a3 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:38.815801 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39855ab6-42f3-477d-8f8b-31fc415c22a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:38.893499 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-39855ab6-42f3-477d-8f8b-31fc415c22a3 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:38.899017 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-39855ab6-42f3-477d-8f8b-31fc415c22a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:38.973579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 910/1822] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:38.992418 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac34151-828a-4851-a930-6fda630db265 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:38.993977 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-cac34151-828a-4851-a930-6fda630db265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:39.077442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 913/1823] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:32:39.078440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39855ab6-42f3-477d-8f8b-31fc415c22a3 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:39.089397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-39855ab6-42f3-477d-8f8b-31fc415c22a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:39.121500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39855ab6-42f3-477d-8f8b-31fc415c22a3 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:39.122705 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:39.123757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:39.124203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:39.124488 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc admin admin] PATH_INFO: `/v3/projects/09f0a4400b414d729e1eeb3714cf5bd5` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:39.125735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=09f0a4400b414d729e1eeb3714cf5bd5)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:39.127925 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:39.131746 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac34151-828a-4851-a930-6fda630db265 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:39.196114 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30252c4-6d49-410f-8fb1-27a75262a3bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:39.197548 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30252c4-6d49-410f-8fb1-27a75262a3bc service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:39.197958 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30252c4-6d49-410f-8fb1-27a75262a3bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:39.198390 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30252c4-6d49-410f-8fb1-27a75262a3bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:39.267333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30252c4-6d49-410f-8fb1-27a75262a3bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:39.268716 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30252c4-6d49-410f-8fb1-27a75262a3bc service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:39.284221 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 911/1824] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:32:39.284553 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30252c4-6d49-410f-8fb1-27a75262a3bc service cinder] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:39.288651 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b30252c4-6d49-410f-8fb1-27a75262a3bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:39.352152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09f0a4400b414d729e1eeb3714cf5bd5', 'request_id': 'req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:39.399395 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30252c4-6d49-410f-8fb1-27a75262a3bc service cinder] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:39.452659 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09f0a4400b414d729e1eeb3714cf5bd5', 'request_id': 'req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:39.460540 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09f0a4400b414d729e1eeb3714cf5bd5', 'request_id': 'req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:39.473719 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc admin admin] The token cache is being invalidate because project 09f0a4400b414d729e1eeb3714cf5bd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:39.474097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:39.474899 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:39.476705 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 914/1825] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:39 2022] DELETE /identity/v3/projects/09f0a4400b414d729e1eeb3714cf5bd5 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:39.477309 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:39.478849 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:39.489459 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34774c41-b52d-46e6-974c-f029c7a21779 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:39.490131 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34774c41-b52d-46e6-974c-f029c7a21779 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:39.490512 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34774c41-b52d-46e6-974c-f029c7a21779 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:39.490841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34774c41-b52d-46e6-974c-f029c7a21779 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:39.492546 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-34774c41-b52d-46e6-974c-f029c7a21779 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:39.548166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bde0ef-2b0b-4509-b67a-b205e93e02cc admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:39.552976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bc18d-3389-4a57-9004-3153a955177a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:39.553886 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bc18d-3389-4a57-9004-3153a955177a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:39.554171 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bc18d-3389-4a57-9004-3153a955177a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:39.554492 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bc18d-3389-4a57-9004-3153a955177a admin admin] PATH_INFO: `/v3/users/37fee75a0acc4c3f8ff81793c3fb7b39` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:39.557830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6bc18d-3389-4a57-9004-3153a955177a admin admin] RBAC: Authorizing `identity:delete_user(user_id=37fee75a0acc4c3f8ff81793c3fb7b39)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:39.559896 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6bc18d-3389-4a57-9004-3153a955177a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:39.565419 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34774c41-b52d-46e6-974c-f029c7a21779 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:39.565419 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34774c41-b52d-46e6-974c-f029c7a21779 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:39.574065 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 912/1826] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:39.575601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a88137-7256-4e76-8b0a-f1b8f8fabbb5 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:39.576222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a88137-7256-4e76-8b0a-f1b8f8fabbb5 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:39.576506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a88137-7256-4e76-8b0a-f1b8f8fabbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:39.608044 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-e1a88137-7256-4e76-8b0a-f1b8f8fabbb5 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:39.608044 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e1a88137-7256-4e76-8b0a-f1b8f8fabbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:39.616115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 913/1827] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:39.624855 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdd5fe-a955-46fd-9eba-b903caf1c121 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:39.625147 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdd5fe-a955-46fd-9eba-b903caf1c121 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:39.625425 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdd5fe-a955-46fd-9eba-b903caf1c121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:39.657418 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-ddfdd5fe-a955-46fd-9eba-b903caf1c121 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:39.659423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ddfdd5fe-a955-46fd-9eba-b903caf1c121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:39.670376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 914/1828] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:32:39.675505 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdd5fe-a955-46fd-9eba-b903caf1c121 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:39.689146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdd5fe-a955-46fd-9eba-b903caf1c121 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:39.695258 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba091a-ad43-40f7-bb97-1e75efde7773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:39.696034 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba091a-ad43-40f7-bb97-1e75efde7773 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:39.696334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba091a-ad43-40f7-bb97-1e75efde7773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:39.698392 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba091a-ad43-40f7-bb97-1e75efde7773 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:39.699877 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba091a-ad43-40f7-bb97-1e75efde7773 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:32:39.700343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba091a-ad43-40f7-bb97-1e75efde7773 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:39.702579 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba091a-ad43-40f7-bb97-1e75efde7773 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:39.718834 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 915/1829] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:32:39.729708 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba091a-ad43-40f7-bb97-1e75efde7773 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:39.740716 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba091a-ad43-40f7-bb97-1e75efde7773 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:39.749733 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ac068-b47e-434d-a3d3-34129e723372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:39.749733 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ac068-b47e-434d-a3d3-34129e723372 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:39.750747 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ac068-b47e-434d-a3d3-34129e723372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:39.751067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ac068-b47e-434d-a3d3-34129e723372 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:39.751839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ac068-b47e-434d-a3d3-34129e723372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:39.754059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ac068-b47e-434d-a3d3-34129e723372 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:39.795841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-6c6bc18d-3389-4a57-9004-3153a955177a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37fee75a0acc4c3f8ff81793c3fb7b39'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:39.860999 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 916/1830] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:39 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:32:39.865331 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ac068-b47e-434d-a3d3-34129e723372 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:39.878701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-6c6bc18d-3389-4a57-9004-3153a955177a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37fee75a0acc4c3f8ff81793c3fb7b39'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:39.879722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:39.879879 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-6c6bc18d-3389-4a57-9004-3153a955177a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37fee75a0acc4c3f8ff81793c3fb7b39'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:39.902315 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-6c6bc18d-3389-4a57-9004-3153a955177a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37fee75a0acc4c3f8ff81793c3fb7b39'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:39.923653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-6c6bc18d-3389-4a57-9004-3153a955177a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37fee75a0acc4c3f8ff81793c3fb7b39'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:39.937361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 915/1831] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:39 2022] DELETE /identity/v3/users/37fee75a0acc4c3f8ff81793c3fb7b39 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:39.938877 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8d116-a2e2-4db9-ad9e-aee6a5f9e766 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:39.939228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8d116-a2e2-4db9-ad9e-aee6a5f9e766 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:39.939680 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8d116-a2e2-4db9-ad9e-aee6a5f9e766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:39.966282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-92a8d116-a2e2-4db9-ad9e-aee6a5f9e766 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:39.967846 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-92a8d116-a2e2-4db9-ad9e-aee6a5f9e766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:40.007388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ac068-b47e-434d-a3d3-34129e723372 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:40.072879 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 916/1832] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:40.102425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8d116-a2e2-4db9-ad9e-aee6a5f9e766 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:40.106995 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-92a8d116-a2e2-4db9-ad9e-aee6a5f9e766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:40.112694 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf9f67-efa5-4f3a-9adb-21d0597e3004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:40.113472 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf9f67-efa5-4f3a-9adb-21d0597e3004 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:40.113849 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf9f67-efa5-4f3a-9adb-21d0597e3004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:40.114219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf9f67-efa5-4f3a-9adb-21d0597e3004 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:40.115426 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf9f67-efa5-4f3a-9adb-21d0597e3004 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:40.117785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf9f67-efa5-4f3a-9adb-21d0597e3004 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:40.198846 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8d116-a2e2-4db9-ad9e-aee6a5f9e766 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:40.204696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d238a-3f3d-4c53-b313-83c9c1588d8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:40.205682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d238a-3f3d-4c53-b313-83c9c1588d8d service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:40.206117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d238a-3f3d-4c53-b313-83c9c1588d8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:40.206638 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d238a-3f3d-4c53-b313-83c9c1588d8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:40.226514 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d238a-3f3d-4c53-b313-83c9c1588d8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:40.229644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d238a-3f3d-4c53-b313-83c9c1588d8d service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:40.257774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 917/1833] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:39 2022] POST /identity/v3/users => generated 291 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:40.261208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf9f67-efa5-4f3a-9adb-21d0597e3004 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:40.263117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 917/1834] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:32:40.265170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-71cf9f67-efa5-4f3a-9adb-21d0597e3004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:40.283135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf9f67-efa5-4f3a-9adb-21d0597e3004 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:40.298505 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc190a-8d98-406c-a4de-313a31f2fbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:40.299640 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc190a-8d98-406c-a4de-313a31f2fbc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:40.300115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc190a-8d98-406c-a4de-313a31f2fbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:40.300657 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc190a-8d98-406c-a4de-313a31f2fbc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:40.303142 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc190a-8d98-406c-a4de-313a31f2fbc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:40.305896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc190a-8d98-406c-a4de-313a31f2fbc0 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:40.316441 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d238a-3f3d-4c53-b313-83c9c1588d8d service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:40.321217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-3a1d238a-3f3d-4c53-b313-83c9c1588d8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:40.356221 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 918/1835] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:40.362232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc190a-8d98-406c-a4de-313a31f2fbc0 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:40.376439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc190a-8d98-406c-a4de-313a31f2fbc0 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:40.384222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52d5b7-d91d-4c9e-82a7-8cc0ca6d04ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:40.385002 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52d5b7-d91d-4c9e-82a7-8cc0ca6d04ab admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:40.385347 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52d5b7-d91d-4c9e-82a7-8cc0ca6d04ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:40.385991 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52d5b7-d91d-4c9e-82a7-8cc0ca6d04ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:40.387253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52d5b7-d91d-4c9e-82a7-8cc0ca6d04ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:40.390118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52d5b7-d91d-4c9e-82a7-8cc0ca6d04ab admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:40.415333 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d238a-3f3d-4c53-b313-83c9c1588d8d service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:40.421526 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 919/1836] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:40.427422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52d5b7-d91d-4c9e-82a7-8cc0ca6d04ab admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:40.435298 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a6a8c-5568-4596-9bae-9b1f46dd700d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:40.436195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a6a8c-5568-4596-9bae-9b1f46dd700d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:40.436480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a6a8c-5568-4596-9bae-9b1f46dd700d admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:40.436754 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a6a8c-5568-4596-9bae-9b1f46dd700d admin admin] PATH_INFO: `/v3/projects/9994bd5e2ff64edc91b18b8c329da824` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:40.440288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a6a8c-5568-4596-9bae-9b1f46dd700d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9994bd5e2ff64edc91b18b8c329da824)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:40.442224 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a6a8c-5568-4596-9bae-9b1f46dd700d admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:40.450425 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52d5b7-d91d-4c9e-82a7-8cc0ca6d04ab admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:40.489403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754f5cc-18a8-41d8-b954-f7e3a014b5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:40.496858 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754f5cc-18a8-41d8-b954-f7e3a014b5eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:40.497675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754f5cc-18a8-41d8-b954-f7e3a014b5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:40.501060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754f5cc-18a8-41d8-b954-f7e3a014b5eb admin admin] PATH_INFO: `/v3/projects/9b9aa60376864f828dbae6dbfde97278/users/f230e8f931e746dcb44aeb7bebf271bd/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:40.523131 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754f5cc-18a8-41d8-b954-f7e3a014b5eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b9aa60376864f828dbae6dbfde97278, user_id=f230e8f931e746dcb44aeb7bebf271bd, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:40.526948 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754f5cc-18a8-41d8-b954-f7e3a014b5eb admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:40.643223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 920/1837] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:40 2022] PUT /identity/v3/projects/9b9aa60376864f828dbae6dbfde97278/users/f230e8f931e746dcb44aeb7bebf271bd/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:40.648270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff664a97-42b1-4e78-8671-05e138442674 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:40.648587 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff664a97-42b1-4e78-8671-05e138442674 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:40.648848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff664a97-42b1-4e78-8671-05e138442674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:40.671077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-001a6a8c-5568-4596-9bae-9b1f46dd700d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9994bd5e2ff64edc91b18b8c329da824', 'request_id': 'req-001a6a8c-5568-4596-9bae-9b1f46dd700d'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:40.689318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-ff664a97-42b1-4e78-8671-05e138442674 None None] MFA Rules not processed for user `f230e8f931e746dcb44aeb7bebf271bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:40.690744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ff664a97-42b1-4e78-8671-05e138442674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:40.786808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 921/1838] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:32:40.794061 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff664a97-42b1-4e78-8671-05e138442674 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:40.812492 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-001a6a8c-5568-4596-9bae-9b1f46dd700d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9994bd5e2ff64edc91b18b8c329da824', 'request_id': 'req-001a6a8c-5568-4596-9bae-9b1f46dd700d'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:40.824203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-001a6a8c-5568-4596-9bae-9b1f46dd700d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9994bd5e2ff64edc91b18b8c329da824', 'request_id': 'req-001a6a8c-5568-4596-9bae-9b1f46dd700d'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:40.840141 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-001a6a8c-5568-4596-9bae-9b1f46dd700d admin admin] The token cache is being invalidate because project 9994bd5e2ff64edc91b18b8c329da824 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:40.840894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-001a6a8c-5568-4596-9bae-9b1f46dd700d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:40.842192 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-001a6a8c-5568-4596-9bae-9b1f46dd700d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:40.845563 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 918/1839] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:40 2022] DELETE /identity/v3/projects/9994bd5e2ff64edc91b18b8c329da824 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:40.849966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a6a8c-5568-4596-9bae-9b1f46dd700d admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:40.852671 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-001a6a8c-5568-4596-9bae-9b1f46dd700d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:40.882785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff664a97-42b1-4e78-8671-05e138442674 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:40.893189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e489d-3759-4bb2-9d2e-5951811b8c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:40.893874 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e489d-3759-4bb2-9d2e-5951811b8c14 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:40.894189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e489d-3759-4bb2-9d2e-5951811b8c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:40.894494 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e489d-3759-4bb2-9d2e-5951811b8c14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:40.895190 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e489d-3759-4bb2-9d2e-5951811b8c14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:40.897851 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e489d-3759-4bb2-9d2e-5951811b8c14 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:40.945089 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a6a8c-5568-4596-9bae-9b1f46dd700d admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:40.949892 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901be731-497a-4f09-b7f5-79249c9a57f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:40.950933 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901be731-497a-4f09-b7f5-79249c9a57f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:40.951091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901be731-497a-4f09-b7f5-79249c9a57f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:40.951437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901be731-497a-4f09-b7f5-79249c9a57f1 admin admin] PATH_INFO: `/v3/users/dd7340eca51346a19ffe1fada6bf7389` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:40.952724 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901be731-497a-4f09-b7f5-79249c9a57f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd7340eca51346a19ffe1fada6bf7389)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:40.954569 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901be731-497a-4f09-b7f5-79249c9a57f1 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:41.015743 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 922/1840] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:40 2022] POST /identity/v3/projects => generated 383 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:32:41.017954 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e489d-3759-4bb2-9d2e-5951811b8c14 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:41.021980 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2a0e489d-3759-4bb2-9d2e-5951811b8c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:41.109383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e489d-3759-4bb2-9d2e-5951811b8c14 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:41.115071 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:41.116034 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:41.116341 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:41.116629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9 admin admin] PATH_INFO: `/v3/projects/73192402244d44739ff3bf04bdd9269b` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:41.117939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73192402244d44739ff3bf04bdd9269b)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:41.120002 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:41.163506 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-901be731-497a-4f09-b7f5-79249c9a57f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd7340eca51346a19ffe1fada6bf7389'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:41.255808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-901be731-497a-4f09-b7f5-79249c9a57f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd7340eca51346a19ffe1fada6bf7389'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:41.262293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-901be731-497a-4f09-b7f5-79249c9a57f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd7340eca51346a19ffe1fada6bf7389'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:41.296411 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-901be731-497a-4f09-b7f5-79249c9a57f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd7340eca51346a19ffe1fada6bf7389'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:41.296721 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73192402244d44739ff3bf04bdd9269b', 'request_id': 'req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:41.306126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-901be731-497a-4f09-b7f5-79249c9a57f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd7340eca51346a19ffe1fada6bf7389'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:41.312875 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 919/1841] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:40 2022] DELETE /identity/v3/users/dd7340eca51346a19ffe1fada6bf7389 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:41.314102 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901be731-497a-4f09-b7f5-79249c9a57f1 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:41.315998 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-901be731-497a-4f09-b7f5-79249c9a57f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:41.343978 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:41.363679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73192402244d44739ff3bf04bdd9269b', 'request_id': 'req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:41.375211 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73192402244d44739ff3bf04bdd9269b', 'request_id': 'req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:41.387958 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9 admin admin] The token cache is being invalidate because project 73192402244d44739ff3bf04bdd9269b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:41.388132 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:41.389861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:41.395197 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 923/1842] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:41 2022] DELETE /identity/v3/projects/73192402244d44739ff3bf04bdd9269b => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:41.396809 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:41.398598 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:41.414118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901be731-497a-4f09-b7f5-79249c9a57f1 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:41.419744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f9132-5cb9-4ad4-84af-e40d38ed8ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:41.420430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f9132-5cb9-4ad4-84af-e40d38ed8ef2 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:41.420783 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f9132-5cb9-4ad4-84af-e40d38ed8ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:41.421707 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f9132-5cb9-4ad4-84af-e40d38ed8ef2 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:41.422459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f9132-5cb9-4ad4-84af-e40d38ed8ef2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:41.424374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f9132-5cb9-4ad4-84af-e40d38ed8ef2 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:41.477922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b14b68-a5fb-4cf0-b823-e1cc97e796a9 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:41.482767 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc700a60-dde2-4348-b3c1-0af29a5c600c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:41.483671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc700a60-dde2-4348-b3c1-0af29a5c600c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:41.483950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc700a60-dde2-4348-b3c1-0af29a5c600c admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:41.484222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc700a60-dde2-4348-b3c1-0af29a5c600c admin admin] PATH_INFO: `/v3/users/f537bcffc65b467ba11c9b758cdda68e` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:41.485549 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc700a60-dde2-4348-b3c1-0af29a5c600c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f537bcffc65b467ba11c9b758cdda68e)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:41.487630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc700a60-dde2-4348-b3c1-0af29a5c600c admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:41.512935 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 920/1843] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:41 2022] POST /identity/v3/users => generated 298 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:41.517268 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f9132-5cb9-4ad4-84af-e40d38ed8ef2 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:41.519238 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ba8f9132-5cb9-4ad4-84af-e40d38ed8ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:41.533490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f9132-5cb9-4ad4-84af-e40d38ed8ef2 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:41.539695 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6408900-0178-48c1-ab14-92c43d60ea05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:41.540370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6408900-0178-48c1-ab14-92c43d60ea05 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:41.540655 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6408900-0178-48c1-ab14-92c43d60ea05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:41.540934 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6408900-0178-48c1-ab14-92c43d60ea05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:41.541679 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6408900-0178-48c1-ab14-92c43d60ea05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:41.543298 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6408900-0178-48c1-ab14-92c43d60ea05 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:41.553782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 921/1844] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:32:41.557996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6408900-0178-48c1-ab14-92c43d60ea05 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:41.567848 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6408900-0178-48c1-ab14-92c43d60ea05 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:41.573004 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697fe8b-cd2a-4431-b351-8b1810719fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:41.573899 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697fe8b-cd2a-4431-b351-8b1810719fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:41.580915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697fe8b-cd2a-4431-b351-8b1810719fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:41.580915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697fe8b-cd2a-4431-b351-8b1810719fe2 admin admin] PATH_INFO: `/v3/projects/2b31c6fca38e4599a06fe861ce97db8c/users/553c17eefe1f4132b885e9111c127152/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:41.591254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697fe8b-cd2a-4431-b351-8b1810719fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b31c6fca38e4599a06fe861ce97db8c, user_id=553c17eefe1f4132b885e9111c127152, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:41.594139 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697fe8b-cd2a-4431-b351-8b1810719fe2 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:41.706385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 922/1845] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:41 2022] PUT /identity/v3/projects/2b31c6fca38e4599a06fe861ce97db8c/users/553c17eefe1f4132b885e9111c127152/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:41.710383 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697fe8b-cd2a-4431-b351-8b1810719fe2 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:41.772521 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-fc700a60-dde2-4348-b3c1-0af29a5c600c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f537bcffc65b467ba11c9b758cdda68e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:41.798769 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697fe8b-cd2a-4431-b351-8b1810719fe2 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:41.804023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47ee22-3369-4868-8820-4a52aa2d8489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:41.804780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47ee22-3369-4868-8820-4a52aa2d8489 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:41.805169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47ee22-3369-4868-8820-4a52aa2d8489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:41.805516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47ee22-3369-4868-8820-4a52aa2d8489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:41.806858 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47ee22-3369-4868-8820-4a52aa2d8489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:41.808514 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47ee22-3369-4868-8820-4a52aa2d8489 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:41.822298 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 923/1846] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:32:41.825365 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47ee22-3369-4868-8820-4a52aa2d8489 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:41.835789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47ee22-3369-4868-8820-4a52aa2d8489 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:41.840916 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-fc700a60-dde2-4348-b3c1-0af29a5c600c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f537bcffc65b467ba11c9b758cdda68e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:41.847348 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8411ce-85ac-41be-8c9e-6ea120e34531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:41.848246 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8411ce-85ac-41be-8c9e-6ea120e34531 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:41.850285 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8411ce-85ac-41be-8c9e-6ea120e34531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:41.850285 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8411ce-85ac-41be-8c9e-6ea120e34531 admin admin] PATH_INFO: `/v3/projects/2b31c6fca38e4599a06fe861ce97db8c/users/553c17eefe1f4132b885e9111c127152/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:41.850873 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8411ce-85ac-41be-8c9e-6ea120e34531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b31c6fca38e4599a06fe861ce97db8c, user_id=553c17eefe1f4132b885e9111c127152, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:41.853472 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-fc700a60-dde2-4348-b3c1-0af29a5c600c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f537bcffc65b467ba11c9b758cdda68e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:41.853956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8411ce-85ac-41be-8c9e-6ea120e34531 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:41.854403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-fc700a60-dde2-4348-b3c1-0af29a5c600c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f537bcffc65b467ba11c9b758cdda68e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:41.865295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 924/1847] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:41 2022] PUT /identity/v3/projects/2b31c6fca38e4599a06fe861ce97db8c/users/553c17eefe1f4132b885e9111c127152/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:41.877081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e621bf-b1fe-4cbf-a3d3-fe1e4d4e0e69 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:41.877377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e621bf-b1fe-4cbf-a3d3-fe1e4d4e0e69 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:41.877667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e621bf-b1fe-4cbf-a3d3-fe1e4d4e0e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:41.878672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-fc700a60-dde2-4348-b3c1-0af29a5c600c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f537bcffc65b467ba11c9b758cdda68e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:41.887174 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 924/1848] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:41 2022] DELETE /identity/v3/users/f537bcffc65b467ba11c9b758cdda68e => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:41.922884 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc700a60-dde2-4348-b3c1-0af29a5c600c admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:41.924169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-a1e621bf-b1fe-4cbf-a3d3-fe1e4d4e0e69 None None] MFA Rules not processed for user `553c17eefe1f4132b885e9111c127152`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:41.925633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a1e621bf-b1fe-4cbf-a3d3-fe1e4d4e0e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:41.927829 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-fc700a60-dde2-4348-b3c1-0af29a5c600c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:42.041224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc700a60-dde2-4348-b3c1-0af29a5c600c admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:42.047437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfac48-1ad3-422d-9e72-9a9ba24b3bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:42.048453 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfac48-1ad3-422d-9e72-9a9ba24b3bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:42.049086 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfac48-1ad3-422d-9e72-9a9ba24b3bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:42.049506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfac48-1ad3-422d-9e72-9a9ba24b3bc7 admin admin] PATH_INFO: `/v3/projects/e826253d494d4bbf88f60658ea0e13b6` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:42.051535 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfac48-1ad3-422d-9e72-9a9ba24b3bc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e826253d494d4bbf88f60658ea0e13b6)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:42.054255 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfac48-1ad3-422d-9e72-9a9ba24b3bc7 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:42.068594 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 925/1849] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:32:42.086797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e621bf-b1fe-4cbf-a3d3-fe1e4d4e0e69 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:42.098290 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a1e621bf-b1fe-4cbf-a3d3-fe1e4d4e0e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:42.201821 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e621bf-b1fe-4cbf-a3d3-fe1e4d4e0e69 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:42.204639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e1bfac48-1ad3-422d-9e72-9a9ba24b3bc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e826253d494d4bbf88f60658ea0e13b6', 'request_id': 'req-e1bfac48-1ad3-422d-9e72-9a9ba24b3bc7'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:42.273800 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d887d4-30de-46c5-b02a-5d8519e38708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:42.274487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d887d4-30de-46c5-b02a-5d8519e38708 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:42.274785 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d887d4-30de-46c5-b02a-5d8519e38708 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:42.275053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d887d4-30de-46c5-b02a-5d8519e38708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:42.289037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e1bfac48-1ad3-422d-9e72-9a9ba24b3bc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e826253d494d4bbf88f60658ea0e13b6', 'request_id': 'req-e1bfac48-1ad3-422d-9e72-9a9ba24b3bc7'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:42.310490 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e1bfac48-1ad3-422d-9e72-9a9ba24b3bc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e826253d494d4bbf88f60658ea0e13b6', 'request_id': 'req-e1bfac48-1ad3-422d-9e72-9a9ba24b3bc7'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:42.335907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e1bfac48-1ad3-422d-9e72-9a9ba24b3bc7 admin admin] The token cache is being invalidate because project e826253d494d4bbf88f60658ea0e13b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:42.336526 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e1bfac48-1ad3-422d-9e72-9a9ba24b3bc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:42.338696 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e1bfac48-1ad3-422d-9e72-9a9ba24b3bc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:42.341451 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 925/1850] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:41 2022] DELETE /identity/v3/projects/e826253d494d4bbf88f60658ea0e13b6 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:42.347242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bcb9e-fc99-4d30-adaf-1ebd2ef058c2 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:42.347669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bcb9e-fc99-4d30-adaf-1ebd2ef058c2 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:42.348046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bcb9e-fc99-4d30-adaf-1ebd2ef058c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:42.365114 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d887d4-30de-46c5-b02a-5d8519e38708 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:42.367148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d887d4-30de-46c5-b02a-5d8519e38708 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:42.379929 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-fe7bcb9e-fc99-4d30-adaf-1ebd2ef058c2 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:42.381103 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-fe7bcb9e-fc99-4d30-adaf-1ebd2ef058c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:42.381346 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 926/1851] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:42.469202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 926/1852] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:32:42.475597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c41e4-92bd-44f8-9ddf-a08c0abbf02c None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:42.476530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c41e4-92bd-44f8-9ddf-a08c0abbf02c None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:42.478781 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c41e4-92bd-44f8-9ddf-a08c0abbf02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:42.499696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-790c41e4-92bd-44f8-9ddf-a08c0abbf02c None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:42.500937 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-790c41e4-92bd-44f8-9ddf-a08c0abbf02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:42.515403 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 927/1853] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:42.515700 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bcb9e-fc99-4d30-adaf-1ebd2ef058c2 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:42.533362 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bcb9e-fc99-4d30-adaf-1ebd2ef058c2 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:42.542312 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fe982-803c-439c-8caa-ab4aa82f898f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:42.544033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fe982-803c-439c-8caa-ab4aa82f898f admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:42.544606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fe982-803c-439c-8caa-ab4aa82f898f admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:42.545100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fe982-803c-439c-8caa-ab4aa82f898f admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:42.546763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199fe982-803c-439c-8caa-ab4aa82f898f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:32:42.547194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199fe982-803c-439c-8caa-ab4aa82f898f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:42.550396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199fe982-803c-439c-8caa-ab4aa82f898f admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:42.579364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 927/1854] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:32:42.584922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fe982-803c-439c-8caa-ab4aa82f898f admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:42.585370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c41e4-92bd-44f8-9ddf-a08c0abbf02c None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:42.586852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-199fe982-803c-439c-8caa-ab4aa82f898f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:42.600701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c41e4-92bd-44f8-9ddf-a08c0abbf02c None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:42.602542 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fe982-803c-439c-8caa-ab4aa82f898f admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:42.607783 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d2624-83b6-48f5-b261-7a68291111c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:42.608483 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d2624-83b6-48f5-b261-7a68291111c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:42.608766 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d2624-83b6-48f5-b261-7a68291111c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:42.609047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d2624-83b6-48f5-b261-7a68291111c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:42.611680 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665947a-202a-4e70-9073-fff76656a7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:42.614254 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-981d2624-83b6-48f5-b261-7a68291111c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:42.614375 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665947a-202a-4e70-9073-fff76656a7a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:42.614375 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665947a-202a-4e70-9073-fff76656a7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:42.614375 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665947a-202a-4e70-9073-fff76656a7a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:42.614740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e665947a-202a-4e70-9073-fff76656a7a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:42.618368 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e665947a-202a-4e70-9073-fff76656a7a5 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:42.692494 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d2624-83b6-48f5-b261-7a68291111c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:42.694370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d2624-83b6-48f5-b261-7a68291111c3 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:42.707480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 928/1855] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:42.753454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 928/1856] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:42 2022] POST /identity/v3/projects => generated 367 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:42.754770 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d2624-83b6-48f5-b261-7a68291111c3 service cinder] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:42.770835 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d2624-83b6-48f5-b261-7a68291111c3 service cinder] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:42.861290 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665947a-202a-4e70-9073-fff76656a7a5 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:42.866340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e665947a-202a-4e70-9073-fff76656a7a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:42.894958 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7b424-a4b3-4584-a034-abfa36c23b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:42.896469 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7b424-a4b3-4584-a034-abfa36c23b13 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:42.896903 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7b424-a4b3-4584-a034-abfa36c23b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:42.897327 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7b424-a4b3-4584-a034-abfa36c23b13 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:42.898313 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7b424-a4b3-4584-a034-abfa36c23b13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:42.901444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7b424-a4b3-4584-a034-abfa36c23b13 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:42.907180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:42.992021 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665947a-202a-4e70-9073-fff76656a7a5 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:43.000135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:43.001428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:43.001428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:43.001732 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced admin admin] PATH_INFO: `/v3/projects/297ad5303700438686e655c6df1fe40a` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:43.011931 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced admin admin] RBAC: Authorizing `identity:delete_project(project_id=297ad5303700438686e655c6df1fe40a)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:43.013438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 929/1857] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:42 2022] POST /identity/v3/users => generated 284 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:43.017526 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7b424-a4b3-4584-a034-abfa36c23b13 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:43.019687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:43.045517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7b424-a4b3-4584-a034-abfa36c23b13 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:43.051984 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5232ac-0bfb-4aa9-9920-e46ac171cac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:43.052671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5232ac-0bfb-4aa9-9920-e46ac171cac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:43.052956 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5232ac-0bfb-4aa9-9920-e46ac171cac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:43.053240 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5232ac-0bfb-4aa9-9920-e46ac171cac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:43.053973 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5232ac-0bfb-4aa9-9920-e46ac171cac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:43.055638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5232ac-0bfb-4aa9-9920-e46ac171cac5 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:43.067374 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 930/1858] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:32:43.071517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5232ac-0bfb-4aa9-9920-e46ac171cac5 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:43.088545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5232ac-0bfb-4aa9-9920-e46ac171cac5 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:43.096317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2fb43-d68b-43e9-9303-dc350773bacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:43.097925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2fb43-d68b-43e9-9303-dc350773bacd admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:43.098444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2fb43-d68b-43e9-9303-dc350773bacd admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:43.098879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2fb43-d68b-43e9-9303-dc350773bacd admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:43.099627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc2fb43-d68b-43e9-9303-dc350773bacd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:43.101328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc2fb43-d68b-43e9-9303-dc350773bacd admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:43.141164 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 931/1859] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:32:43.147059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2fb43-d68b-43e9-9303-dc350773bacd admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:43.159749 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2fb43-d68b-43e9-9303-dc350773bacd admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:43.166378 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403464fe-1042-4f5d-b2bf-b4be30544648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:43.167568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403464fe-1042-4f5d-b2bf-b4be30544648 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:43.168005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403464fe-1042-4f5d-b2bf-b4be30544648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:43.168481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403464fe-1042-4f5d-b2bf-b4be30544648 admin admin] PATH_INFO: `/v3/projects/b98c6f582c524955a474383d355a7612/users/6482006517154d8b88824e3805c19403/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:43.182070 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '297ad5303700438686e655c6df1fe40a', 'request_id': 'req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:43.199268 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403464fe-1042-4f5d-b2bf-b4be30544648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b98c6f582c524955a474383d355a7612, user_id=6482006517154d8b88824e3805c19403, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:43.201380 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403464fe-1042-4f5d-b2bf-b4be30544648 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:43.309952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '297ad5303700438686e655c6df1fe40a', 'request_id': 'req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:43.318821 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '297ad5303700438686e655c6df1fe40a', 'request_id': 'req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:43.319737 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 932/1860] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:43 2022] PUT /identity/v3/projects/b98c6f582c524955a474383d355a7612/users/6482006517154d8b88824e3805c19403/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:43.340226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced admin admin] The token cache is being invalidate because project 297ad5303700438686e655c6df1fe40a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:43.340780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:43.340910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3b120-3ca7-4a4d-9a31-b21520932dcd None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:43.341300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3b120-3ca7-4a4d-9a31-b21520932dcd None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:43.341495 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:43.341862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3b120-3ca7-4a4d-9a31-b21520932dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:43.343427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 929/1861] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:42 2022] DELETE /identity/v3/projects/297ad5303700438686e655c6df1fe40a => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:43.344224 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:43.345732 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:43.374584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-b7e3b120-3ca7-4a4d-9a31-b21520932dcd None None] MFA Rules not processed for user `6482006517154d8b88824e3805c19403`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:43.375801 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b7e3b120-3ca7-4a4d-9a31-b21520932dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:43.431892 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457aa2b-ccb0-47bc-8576-1d4dcf584ced admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:43.467040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 933/1862] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:32:43.467040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac20094-4f51-4eb1-9be5-7609fecae287 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:43.467040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac20094-4f51-4eb1-9be5-7609fecae287 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:43.467040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac20094-4f51-4eb1-9be5-7609fecae287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:43.490164 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-3ac20094-4f51-4eb1-9be5-7609fecae287 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:43.492105 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-3ac20094-4f51-4eb1-9be5-7609fecae287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:43.507129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ad557-052e-40df-ab72-d4ec5e8d50b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:43.508018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ad557-052e-40df-ab72-d4ec5e8d50b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:43.508304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ad557-052e-40df-ab72-d4ec5e8d50b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:43.508584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ad557-052e-40df-ab72-d4ec5e8d50b2 admin admin] PATH_INFO: `/v3/users/f230e8f931e746dcb44aeb7bebf271bd` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:43.509872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ad557-052e-40df-ab72-d4ec5e8d50b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f230e8f931e746dcb44aeb7bebf271bd)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:43.511841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ad557-052e-40df-ab72-d4ec5e8d50b2 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:43.566325 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 934/1863] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:43.566905 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac20094-4f51-4eb1-9be5-7609fecae287 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:43.568293 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-3ac20094-4f51-4eb1-9be5-7609fecae287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:43.582041 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac20094-4f51-4eb1-9be5-7609fecae287 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:43.587940 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f6875-73a0-4cfe-8043-b33fbcf6d1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:43.595852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f6875-73a0-4cfe-8043-b33fbcf6d1c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:43.595852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f6875-73a0-4cfe-8043-b33fbcf6d1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:43.595852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f6875-73a0-4cfe-8043-b33fbcf6d1c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:43.595852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f6875-73a0-4cfe-8043-b33fbcf6d1c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:43.595852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f6875-73a0-4cfe-8043-b33fbcf6d1c7 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:43.731222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 935/1864] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:43 2022] POST /identity/v3/projects => generated 369 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:43.732456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78974b4-05b3-4835-b256-b3cf05cd9aeb None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:43.732757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78974b4-05b3-4835-b256-b3cf05cd9aeb None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:43.733038 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78974b4-05b3-4835-b256-b3cf05cd9aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:43.754621 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-071ad557-052e-40df-ab72-d4ec5e8d50b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f230e8f931e746dcb44aeb7bebf271bd'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:43.758788 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-f78974b4-05b3-4835-b256-b3cf05cd9aeb None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:43.760869 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f78974b4-05b3-4835-b256-b3cf05cd9aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:43.824193 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 936/1865] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:43.825082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78974b4-05b3-4835-b256-b3cf05cd9aeb None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:43.834379 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78974b4-05b3-4835-b256-b3cf05cd9aeb None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:43.837774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-071ad557-052e-40df-ab72-d4ec5e8d50b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f230e8f931e746dcb44aeb7bebf271bd'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:43.838652 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-071ad557-052e-40df-ab72-d4ec5e8d50b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f230e8f931e746dcb44aeb7bebf271bd'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:43.840185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f000b2-0567-4c54-b02f-8eb04acccfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:43.841112 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f000b2-0567-4c54-b02f-8eb04acccfb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:43.841448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f000b2-0567-4c54-b02f-8eb04acccfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:43.841665 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f000b2-0567-4c54-b02f-8eb04acccfb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:43.842814 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f000b2-0567-4c54-b02f-8eb04acccfb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:43.848529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f000b2-0567-4c54-b02f-8eb04acccfb4 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:43.859489 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-071ad557-052e-40df-ab72-d4ec5e8d50b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f230e8f931e746dcb44aeb7bebf271bd'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:43.867717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-071ad557-052e-40df-ab72-d4ec5e8d50b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f230e8f931e746dcb44aeb7bebf271bd'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:43.875208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 930/1866] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:43 2022] DELETE /identity/v3/users/f230e8f931e746dcb44aeb7bebf271bd => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:43.876031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ad557-052e-40df-ab72-d4ec5e8d50b2 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:43.877520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-071ad557-052e-40df-ab72-d4ec5e8d50b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:43.927561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 937/1867] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:43 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:43.927561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af808a9-6898-4251-af2e-aec88448cbc1 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:43.927561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af808a9-6898-4251-af2e-aec88448cbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:43.927561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af808a9-6898-4251-af2e-aec88448cbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:43.947095 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-9af808a9-6898-4251-af2e-aec88448cbc1 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:43.947505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ad557-052e-40df-ab72-d4ec5e8d50b2 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:43.948433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9af808a9-6898-4251-af2e-aec88448cbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:43.953665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977b6b2-dcfa-4cfe-920d-722484aff997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:43.954401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977b6b2-dcfa-4cfe-920d-722484aff997 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:43.954675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977b6b2-dcfa-4cfe-920d-722484aff997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:43.954955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977b6b2-dcfa-4cfe-920d-722484aff997 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:43.955796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977b6b2-dcfa-4cfe-920d-722484aff997 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:32:43.956282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977b6b2-dcfa-4cfe-920d-722484aff997 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:43.957608 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 938/1868] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:43.958851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977b6b2-dcfa-4cfe-920d-722484aff997 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:43.961598 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af808a9-6898-4251-af2e-aec88448cbc1 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:43.963764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9af808a9-6898-4251-af2e-aec88448cbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:43.971057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:43.976197 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af808a9-6898-4251-af2e-aec88448cbc1 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:43.977288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 931/1869] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:32:43.981589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f61899-177e-4f07-bf77-a8166c065356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:43.981806 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977b6b2-dcfa-4cfe-920d-722484aff997 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:43.982220 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f61899-177e-4f07-bf77-a8166c065356 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:43.982531 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f61899-177e-4f07-bf77-a8166c065356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:43.982922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f61899-177e-4f07-bf77-a8166c065356 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:43.983537 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f61899-177e-4f07-bf77-a8166c065356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:43.985201 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f61899-177e-4f07-bf77-a8166c065356 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:43.994348 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977b6b2-dcfa-4cfe-920d-722484aff997 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:43.999387 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 939/1870] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:43.999387 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f61899-177e-4f07-bf77-a8166c065356 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:43.999387 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-23f61899-177e-4f07-bf77-a8166c065356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:44.007511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949358c9-cb96-41d4-b583-7c2256840e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:44.008201 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949358c9-cb96-41d4-b583-7c2256840e55 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:44.008439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949358c9-cb96-41d4-b583-7c2256840e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:44.014505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949358c9-cb96-41d4-b583-7c2256840e55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:44.014505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949358c9-cb96-41d4-b583-7c2256840e55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:44.014505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949358c9-cb96-41d4-b583-7c2256840e55 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:44.085043 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f61899-177e-4f07-bf77-a8166c065356 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:44.089977 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d269d7-bf29-4f0f-8757-36f1d964fb15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:44.091029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d269d7-bf29-4f0f-8757-36f1d964fb15 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:44.091320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d269d7-bf29-4f0f-8757-36f1d964fb15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:44.091598 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d269d7-bf29-4f0f-8757-36f1d964fb15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:44.106847 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d269d7-bf29-4f0f-8757-36f1d964fb15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:44.108770 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d269d7-bf29-4f0f-8757-36f1d964fb15 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:44.122216 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 940/1871] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:44.123178 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d269d7-bf29-4f0f-8757-36f1d964fb15 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:44.136027 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d269d7-bf29-4f0f-8757-36f1d964fb15 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:44.142484 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2987f-58b8-4d3d-a690-8ac5258bbb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:44.143461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2987f-58b8-4d3d-a690-8ac5258bbb84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:44.143932 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2987f-58b8-4d3d-a690-8ac5258bbb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:44.144223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2987f-58b8-4d3d-a690-8ac5258bbb84 admin admin] PATH_INFO: `/v3/projects/f97c36ccb1054e4a9d9a2b90760c19ad/users/4590129e58334bb1a02eeecd1be342c1/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:44.179348 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2987f-58b8-4d3d-a690-8ac5258bbb84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f97c36ccb1054e4a9d9a2b90760c19ad, user_id=4590129e58334bb1a02eeecd1be342c1, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:44.180162 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 932/1872] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:43 2022] POST /identity/v3/projects => generated 367 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:44.181645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949358c9-cb96-41d4-b583-7c2256840e55 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:44.187711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2987f-58b8-4d3d-a690-8ac5258bbb84 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:44.275795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949358c9-cb96-41d4-b583-7c2256840e55 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:44.285082 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622ad00-c15e-4e93-bc1d-0eaa9dc6526d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:44.285897 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622ad00-c15e-4e93-bc1d-0eaa9dc6526d admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:44.286191 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622ad00-c15e-4e93-bc1d-0eaa9dc6526d admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:44.286683 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622ad00-c15e-4e93-bc1d-0eaa9dc6526d admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:44.287692 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622ad00-c15e-4e93-bc1d-0eaa9dc6526d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:44.289666 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622ad00-c15e-4e93-bc1d-0eaa9dc6526d admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:44.324168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 941/1873] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:44 2022] PUT /identity/v3/projects/f97c36ccb1054e4a9d9a2b90760c19ad/users/4590129e58334bb1a02eeecd1be342c1/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:44.328517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2987f-58b8-4d3d-a690-8ac5258bbb84 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:44.399638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2987f-58b8-4d3d-a690-8ac5258bbb84 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:44.405152 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb505c2-d215-4c29-9e95-cbebbb713b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:44.406135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb505c2-d215-4c29-9e95-cbebbb713b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:44.406526 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb505c2-d215-4c29-9e95-cbebbb713b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:44.406839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb505c2-d215-4c29-9e95-cbebbb713b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:44.407617 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb505c2-d215-4c29-9e95-cbebbb713b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:44.409500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb505c2-d215-4c29-9e95-cbebbb713b12 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:44.423551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 942/1874] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:44.425640 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 933/1875] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:44 2022] POST /identity/v3/users => generated 284 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:44.428442 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb505c2-d215-4c29-9e95-cbebbb713b12 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:44.430589 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622ad00-c15e-4e93-bc1d-0eaa9dc6526d admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:44.444803 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622ad00-c15e-4e93-bc1d-0eaa9dc6526d admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:44.447473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb505c2-d215-4c29-9e95-cbebbb713b12 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:44.450846 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c59f5a-46d4-4253-b6c0-ef8a7ab2a3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:44.452559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c59f5a-46d4-4253-b6c0-ef8a7ab2a3e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:44.453274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c59f5a-46d4-4253-b6c0-ef8a7ab2a3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:44.453817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c59f5a-46d4-4253-b6c0-ef8a7ab2a3e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:44.454813 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c59f5a-46d4-4253-b6c0-ef8a7ab2a3e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:44.456953 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c59f5a-46d4-4253-b6c0-ef8a7ab2a3e4 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:44.478517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb76b40-d549-4ed1-aed8-0162a692a0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:44.480839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb76b40-d549-4ed1-aed8-0162a692a0ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:44.480839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb76b40-d549-4ed1-aed8-0162a692a0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:44.481459 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb76b40-d549-4ed1-aed8-0162a692a0ca admin admin] PATH_INFO: `/v3/projects/f97c36ccb1054e4a9d9a2b90760c19ad/users/4590129e58334bb1a02eeecd1be342c1/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:44.481603 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 934/1876] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:32:44.482989 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c59f5a-46d4-4253-b6c0-ef8a7ab2a3e4 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:44.492299 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb76b40-d549-4ed1-aed8-0162a692a0ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=f97c36ccb1054e4a9d9a2b90760c19ad, user_id=4590129e58334bb1a02eeecd1be342c1, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:44.497786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb76b40-d549-4ed1-aed8-0162a692a0ca admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:44.503569 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c59f5a-46d4-4253-b6c0-ef8a7ab2a3e4 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:44.513615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c072d-b536-4942-8595-39a82e88a31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:44.514346 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c072d-b536-4942-8595-39a82e88a31f admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:44.514731 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c072d-b536-4942-8595-39a82e88a31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:44.515247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c072d-b536-4942-8595-39a82e88a31f admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:44.516062 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c072d-b536-4942-8595-39a82e88a31f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:44.517735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c072d-b536-4942-8595-39a82e88a31f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:44.520223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 943/1877] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:44 2022] PUT /identity/v3/projects/f97c36ccb1054e4a9d9a2b90760c19ad/users/4590129e58334bb1a02eeecd1be342c1/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:44.543155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74924aba-99fa-4dc2-9c7b-879b81800824 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:44.543654 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74924aba-99fa-4dc2-9c7b-879b81800824 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:44.546615 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74924aba-99fa-4dc2-9c7b-879b81800824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:44.552770 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:44.559332 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 935/1878] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:44.571869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c072d-b536-4942-8595-39a82e88a31f admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:44.587613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c072d-b536-4942-8595-39a82e88a31f admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:44.598962 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-74924aba-99fa-4dc2-9c7b-879b81800824 None None] MFA Rules not processed for user `4590129e58334bb1a02eeecd1be342c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:44.600479 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-74924aba-99fa-4dc2-9c7b-879b81800824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:44.689975 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 944/1879] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:44 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:44.706651 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74924aba-99fa-4dc2-9c7b-879b81800824 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:44.710238 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-74924aba-99fa-4dc2-9c7b-879b81800824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:44.757309 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36a6da-66f9-439f-bbaa-d9aac46b12c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:44.758240 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36a6da-66f9-439f-bbaa-d9aac46b12c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:44.758681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36a6da-66f9-439f-bbaa-d9aac46b12c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:44.759082 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36a6da-66f9-439f-bbaa-d9aac46b12c5 admin admin] PATH_INFO: `/v3/projects/7f89beb8dc8d4f3ba14717e14c527d7b/users/0097a1e1de024ed0830ba5f7acb8c8e1/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:44.785938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36a6da-66f9-439f-bbaa-d9aac46b12c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f89beb8dc8d4f3ba14717e14c527d7b, user_id=0097a1e1de024ed0830ba5f7acb8c8e1, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:44.793383 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36a6da-66f9-439f-bbaa-d9aac46b12c5 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:44.824411 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74924aba-99fa-4dc2-9c7b-879b81800824 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:44.828520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e55bf-6288-446b-ac0a-d0a9f7946a6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:44.829186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e55bf-6288-446b-ac0a-d0a9f7946a6e service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:44.829470 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e55bf-6288-446b-ac0a-d0a9f7946a6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:44.829747 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e55bf-6288-446b-ac0a-d0a9f7946a6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:44.842736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e55bf-6288-446b-ac0a-d0a9f7946a6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:44.844246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e55bf-6288-446b-ac0a-d0a9f7946a6e service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:44.856377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 945/1880] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:44 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:44.932183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 936/1881] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:44 2022] PUT /identity/v3/projects/7f89beb8dc8d4f3ba14717e14c527d7b/users/0097a1e1de024ed0830ba5f7acb8c8e1/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:44.940472 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3293a41-d2f4-482e-8d18-f267eb4c1ade None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:44.941007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3293a41-d2f4-482e-8d18-f267eb4c1ade None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:44.941101 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3293a41-d2f4-482e-8d18-f267eb4c1ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:44.960324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36a6da-66f9-439f-bbaa-d9aac46b12c5 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:44.963104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-3d36a6da-66f9-439f-bbaa-d9aac46b12c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:45.001404 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-b3293a41-d2f4-482e-8d18-f267eb4c1ade None None] MFA Rules not processed for user `0097a1e1de024ed0830ba5f7acb8c8e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:45.003035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b3293a41-d2f4-482e-8d18-f267eb4c1ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:45.092664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36a6da-66f9-439f-bbaa-d9aac46b12c5 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:45.105589 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed629862-b41c-4a7a-a3c8-7c8720f1f069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:45.107130 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed629862-b41c-4a7a-a3c8-7c8720f1f069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.107630 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed629862-b41c-4a7a-a3c8-7c8720f1f069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.108127 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed629862-b41c-4a7a-a3c8-7c8720f1f069 admin admin] PATH_INFO: `/v3/projects/9b9aa60376864f828dbae6dbfde97278` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.110091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed629862-b41c-4a7a-a3c8-7c8720f1f069 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b9aa60376864f828dbae6dbfde97278)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:45.113532 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed629862-b41c-4a7a-a3c8-7c8720f1f069 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:45.121932 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 946/1882] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:45.121932 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3293a41-d2f4-482e-8d18-f267eb4c1ade None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:45.121932 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b3293a41-d2f4-482e-8d18-f267eb4c1ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:45.182321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3293a41-d2f4-482e-8d18-f267eb4c1ade None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:45.186166 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb5a53-8b0a-4033-a1ec-f418e3a69fd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:45.186733 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb5a53-8b0a-4033-a1ec-f418e3a69fd7 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.187017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb5a53-8b0a-4033-a1ec-f418e3a69fd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.187977 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb5a53-8b0a-4033-a1ec-f418e3a69fd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.190547 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-a4fb5a53-8b0a-4033-a1ec-f418e3a69fd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:45.287303 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ed629862-b41c-4a7a-a3c8-7c8720f1f069 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b9aa60376864f828dbae6dbfde97278', 'request_id': 'req-ed629862-b41c-4a7a-a3c8-7c8720f1f069'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:45.293868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fb5a53-8b0a-4033-a1ec-f418e3a69fd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:45.295829 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fb5a53-8b0a-4033-a1ec-f418e3a69fd7 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:45.306091 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 947/1883] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:45.307054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb5a53-8b0a-4033-a1ec-f418e3a69fd7 service cinder] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:45.334655 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb5a53-8b0a-4033-a1ec-f418e3a69fd7 service cinder] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:45.368969 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ed629862-b41c-4a7a-a3c8-7c8720f1f069 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b9aa60376864f828dbae6dbfde97278', 'request_id': 'req-ed629862-b41c-4a7a-a3c8-7c8720f1f069'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:45.382404 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ed629862-b41c-4a7a-a3c8-7c8720f1f069 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b9aa60376864f828dbae6dbfde97278', 'request_id': 'req-ed629862-b41c-4a7a-a3c8-7c8720f1f069'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:45.444536 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ed629862-b41c-4a7a-a3c8-7c8720f1f069 admin admin] The token cache is being invalidate because project 9b9aa60376864f828dbae6dbfde97278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:45.445457 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ed629862-b41c-4a7a-a3c8-7c8720f1f069 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:45.449328 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ed629862-b41c-4a7a-a3c8-7c8720f1f069 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:45.460988 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 937/1884] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:44 2022] DELETE /identity/v3/projects/9b9aa60376864f828dbae6dbfde97278 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:45.465046 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:45.467251 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed629862-b41c-4a7a-a3c8-7c8720f1f069 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:45.472350 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ed629862-b41c-4a7a-a3c8-7c8720f1f069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:45.550475 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2dc26-3d44-4e83-a250-2a95cb5cf332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:45.551324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2dc26-3d44-4e83-a250-2a95cb5cf332 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.551744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2dc26-3d44-4e83-a250-2a95cb5cf332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.552139 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2dc26-3d44-4e83-a250-2a95cb5cf332 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.553493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2dc26-3d44-4e83-a250-2a95cb5cf332 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:45.556920 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2dc26-3d44-4e83-a250-2a95cb5cf332 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:45.609689 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed629862-b41c-4a7a-a3c8-7c8720f1f069 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:45.614901 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b450d6-b7a5-45ee-8104-7d4dc9e98c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:45.615796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b450d6-b7a5-45ee-8104-7d4dc9e98c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.616134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b450d6-b7a5-45ee-8104-7d4dc9e98c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.616436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b450d6-b7a5-45ee-8104-7d4dc9e98c5a admin admin] PATH_INFO: `/v3/users/553c17eefe1f4132b885e9111c127152` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.617650 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b450d6-b7a5-45ee-8104-7d4dc9e98c5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=553c17eefe1f4132b885e9111c127152)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:45.620161 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b450d6-b7a5-45ee-8104-7d4dc9e98c5a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:45.686217 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 948/1885] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:45 2022] POST /identity/v3/projects => generated 367 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:45.690379 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2dc26-3d44-4e83-a250-2a95cb5cf332 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:45.692602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-05f2dc26-3d44-4e83-a250-2a95cb5cf332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:45.762640 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2dc26-3d44-4e83-a250-2a95cb5cf332 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:45.768568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acb8c2-3823-411b-ae8b-c10347295363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:45.769243 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acb8c2-3823-411b-ae8b-c10347295363 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.769525 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acb8c2-3823-411b-ae8b-c10347295363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.769801 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acb8c2-3823-411b-ae8b-c10347295363 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.770525 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91acb8c2-3823-411b-ae8b-c10347295363 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:45.772755 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91acb8c2-3823-411b-ae8b-c10347295363 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:45.870147 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 949/1886] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:45 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:32:45.875132 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acb8c2-3823-411b-ae8b-c10347295363 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:45.886075 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acb8c2-3823-411b-ae8b-c10347295363 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:45.891275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2681f0-3ac4-48a8-ab96-471252c8ec6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:45.892051 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2681f0-3ac4-48a8-ab96-471252c8ec6f admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.892406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2681f0-3ac4-48a8-ab96-471252c8ec6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.892780 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2681f0-3ac4-48a8-ab96-471252c8ec6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.893852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2681f0-3ac4-48a8-ab96-471252c8ec6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:45.895666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2681f0-3ac4-48a8-ab96-471252c8ec6f admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:45.917319 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 950/1887] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:45.921553 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-06b450d6-b7a5-45ee-8104-7d4dc9e98c5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '553c17eefe1f4132b885e9111c127152'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:45.921949 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2681f0-3ac4-48a8-ab96-471252c8ec6f admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:45.935082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2681f0-3ac4-48a8-ab96-471252c8ec6f admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:45.940842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9861d4-6794-4b16-b71b-f244c6fd6e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:45.941737 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9861d4-6794-4b16-b71b-f244c6fd6e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:45.942046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9861d4-6794-4b16-b71b-f244c6fd6e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:45.942410 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9861d4-6794-4b16-b71b-f244c6fd6e8d admin admin] PATH_INFO: `/v3/projects/4a1d4519396f464f9be17dfe76d75aab/users/b48c89db5c56462a8341b049b4a7108f/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:45.961708 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9861d4-6794-4b16-b71b-f244c6fd6e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a1d4519396f464f9be17dfe76d75aab, user_id=b48c89db5c56462a8341b049b4a7108f, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:45.964879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9861d4-6794-4b16-b71b-f244c6fd6e8d admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:46.024210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-06b450d6-b7a5-45ee-8104-7d4dc9e98c5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '553c17eefe1f4132b885e9111c127152'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:46.027283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-06b450d6-b7a5-45ee-8104-7d4dc9e98c5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '553c17eefe1f4132b885e9111c127152'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:46.041392 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-06b450d6-b7a5-45ee-8104-7d4dc9e98c5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '553c17eefe1f4132b885e9111c127152'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:46.049445 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-06b450d6-b7a5-45ee-8104-7d4dc9e98c5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '553c17eefe1f4132b885e9111c127152'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:46.077589 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 938/1888] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:45 2022] DELETE /identity/v3/users/553c17eefe1f4132b885e9111c127152 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:46.083057 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 951/1889] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:45 2022] PUT /identity/v3/projects/4a1d4519396f464f9be17dfe76d75aab/users/b48c89db5c56462a8341b049b4a7108f/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:46.095604 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b450d6-b7a5-45ee-8104-7d4dc9e98c5a admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:46.097196 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-06b450d6-b7a5-45ee-8104-7d4dc9e98c5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:46.257875 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b450d6-b7a5-45ee-8104-7d4dc9e98c5a admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:46.271333 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e25f69-0b43-4a6a-827b-0f3a36b122c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:46.273574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e25f69-0b43-4a6a-827b-0f3a36b122c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:46.274148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e25f69-0b43-4a6a-827b-0f3a36b122c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:46.275599 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e25f69-0b43-4a6a-827b-0f3a36b122c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:46.278683 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e25f69-0b43-4a6a-827b-0f3a36b122c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:46.281830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e25f69-0b43-4a6a-827b-0f3a36b122c7 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:46.308636 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 939/1890] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:46.313222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9861d4-6794-4b16-b71b-f244c6fd6e8d admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:46.506313 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9861d4-6794-4b16-b71b-f244c6fd6e8d admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:46.512370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0072f-12d5-447f-82bb-7e1ac5db89e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:46.513417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0072f-12d5-447f-82bb-7e1ac5db89e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:46.522423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0072f-12d5-447f-82bb-7e1ac5db89e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:46.522423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0072f-12d5-447f-82bb-7e1ac5db89e5 admin admin] PATH_INFO: `/v3/projects/4a1d4519396f464f9be17dfe76d75aab/users/b48c89db5c56462a8341b049b4a7108f/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:46.522423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c0072f-12d5-447f-82bb-7e1ac5db89e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a1d4519396f464f9be17dfe76d75aab, user_id=b48c89db5c56462a8341b049b4a7108f, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:46.522423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c0072f-12d5-447f-82bb-7e1ac5db89e5 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:46.549628 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 952/1891] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:46 2022] PUT /identity/v3/projects/4a1d4519396f464f9be17dfe76d75aab/users/b48c89db5c56462a8341b049b4a7108f/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:46.567273 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b56c2-1395-4663-afbc-19e1aba559b6 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:46.567811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b56c2-1395-4663-afbc-19e1aba559b6 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:46.568443 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b56c2-1395-4663-afbc-19e1aba559b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:46.616918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-8f7b56c2-1395-4663-afbc-19e1aba559b6 None None] MFA Rules not processed for user `b48c89db5c56462a8341b049b4a7108f`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:46.619072 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8f7b56c2-1395-4663-afbc-19e1aba559b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:46.733250 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:46.757995 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 940/1892] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:46 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:46.786516 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0072f-12d5-447f-82bb-7e1ac5db89e5 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:46.788716 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c7c0072f-12d5-447f-82bb-7e1ac5db89e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:46.881054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0072f-12d5-447f-82bb-7e1ac5db89e5 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:46.885239 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd41b1f-9174-422c-863d-9706a7f77817 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:46.885974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd41b1f-9174-422c-863d-9706a7f77817 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:46.886289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd41b1f-9174-422c-863d-9706a7f77817 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:46.886571 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd41b1f-9174-422c-863d-9706a7f77817 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:46.897899 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd41b1f-9174-422c-863d-9706a7f77817 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:46.899819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd41b1f-9174-422c-863d-9706a7f77817 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:46.926306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 953/1893] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:47.066476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b56c2-1395-4663-afbc-19e1aba559b6 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:47.067897 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8f7b56c2-1395-4663-afbc-19e1aba559b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:47.224707 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b56c2-1395-4663-afbc-19e1aba559b6 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:47.253947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd41b1f-9174-422c-863d-9706a7f77817 service cinder] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:47.266960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6a74c-9009-4df3-a179-fb36323bca2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:47.269523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6a74c-9009-4df3-a179-fb36323bca2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:47.269810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6a74c-9009-4df3-a179-fb36323bca2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:47.270357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6a74c-9009-4df3-a179-fb36323bca2f admin admin] PATH_INFO: `/v3/projects/2b31c6fca38e4599a06fe861ce97db8c` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:47.272923 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6a74c-9009-4df3-a179-fb36323bca2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b31c6fca38e4599a06fe861ce97db8c)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:47.273730 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd41b1f-9174-422c-863d-9706a7f77817 service cinder] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:47.275737 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6a74c-9009-4df3-a179-fb36323bca2f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:47.281367 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345862bd-4f00-4785-8e1c-ba604fa09d98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:47.282552 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345862bd-4f00-4785-8e1c-ba604fa09d98 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:47.283087 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345862bd-4f00-4785-8e1c-ba604fa09d98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:47.283615 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345862bd-4f00-4785-8e1c-ba604fa09d98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:47.287102 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-345862bd-4f00-4785-8e1c-ba604fa09d98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:47.382322 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345862bd-4f00-4785-8e1c-ba604fa09d98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:47.384280 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345862bd-4f00-4785-8e1c-ba604fa09d98 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:47.398556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 954/1894] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:47.449374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-32b6a74c-9009-4df3-a179-fb36323bca2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b31c6fca38e4599a06fe861ce97db8c', 'request_id': 'req-32b6a74c-9009-4df3-a179-fb36323bca2f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:47.541069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-32b6a74c-9009-4df3-a179-fb36323bca2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b31c6fca38e4599a06fe861ce97db8c', 'request_id': 'req-32b6a74c-9009-4df3-a179-fb36323bca2f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:47.551186 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-32b6a74c-9009-4df3-a179-fb36323bca2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b31c6fca38e4599a06fe861ce97db8c', 'request_id': 'req-32b6a74c-9009-4df3-a179-fb36323bca2f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:47.580255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-32b6a74c-9009-4df3-a179-fb36323bca2f admin admin] The token cache is being invalidate because project 2b31c6fca38e4599a06fe861ce97db8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:47.581576 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-32b6a74c-9009-4df3-a179-fb36323bca2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:47.583798 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-32b6a74c-9009-4df3-a179-fb36323bca2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:47.588021 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 941/1895] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:47 2022] DELETE /identity/v3/projects/2b31c6fca38e4599a06fe861ce97db8c => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:47.597401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d53223-b3bb-4608-b1e8-37647d916362 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:47.598046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d53223-b3bb-4608-b1e8-37647d916362 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:47.598225 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d53223-b3bb-4608-b1e8-37647d916362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:47.622848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-05d53223-b3bb-4608-b1e8-37647d916362 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:47.624182 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-05d53223-b3bb-4608-b1e8-37647d916362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:47.650081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:47.701784 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 955/1896] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:47.703977 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd178cd-373b-4513-9aa8-d9d39e6fa2ca None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:47.704270 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd178cd-373b-4513-9aa8-d9d39e6fa2ca None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:47.704550 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd178cd-373b-4513-9aa8-d9d39e6fa2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:47.735664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-5cd178cd-373b-4513-9aa8-d9d39e6fa2ca None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:47.737315 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5cd178cd-373b-4513-9aa8-d9d39e6fa2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:47.747507 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 942/1897] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:47.753650 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d53223-b3bb-4608-b1e8-37647d916362 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:47.765718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d53223-b3bb-4608-b1e8-37647d916362 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:47.770480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cfcf7-3de8-4bd2-9629-215170438c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:47.771155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cfcf7-3de8-4bd2-9629-215170438c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:47.771439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cfcf7-3de8-4bd2-9629-215170438c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:47.771722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cfcf7-3de8-4bd2-9629-215170438c3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:47.772463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cfcf7-3de8-4bd2-9629-215170438c3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:32:47.772773 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cfcf7-3de8-4bd2-9629-215170438c3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:47.774437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cfcf7-3de8-4bd2-9629-215170438c3a admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:47.790313 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 956/1898] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:32:47.801252 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd178cd-373b-4513-9aa8-d9d39e6fa2ca None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:47.817460 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd178cd-373b-4513-9aa8-d9d39e6fa2ca None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:47.833209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d96509-dbdf-49b1-8cf5-7eb360df6129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:47.834023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d96509-dbdf-49b1-8cf5-7eb360df6129 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:47.836575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d96509-dbdf-49b1-8cf5-7eb360df6129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:47.837049 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d96509-dbdf-49b1-8cf5-7eb360df6129 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:47.838063 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d96509-dbdf-49b1-8cf5-7eb360df6129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:47.841684 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d96509-dbdf-49b1-8cf5-7eb360df6129 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:47.986117 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:48.002684 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 943/1899] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:47 2022] POST /identity/v3/projects => generated 373 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:48.003977 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cfcf7-3de8-4bd2-9629-215170438c3a admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:48.016714 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cfcf7-3de8-4bd2-9629-215170438c3a admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:48.109401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb440c-8586-422c-bdf9-f0dc5893d0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:48.110135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb440c-8586-422c-bdf9-f0dc5893d0db admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:48.110488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb440c-8586-422c-bdf9-f0dc5893d0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:48.110777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb440c-8586-422c-bdf9-f0dc5893d0db admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:48.111524 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb440c-8586-422c-bdf9-f0dc5893d0db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:48.113548 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb440c-8586-422c-bdf9-f0dc5893d0db admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:48.210683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 957/1900] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:48 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:32:48.216180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d96509-dbdf-49b1-8cf5-7eb360df6129 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:48.233787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d96509-dbdf-49b1-8cf5-7eb360df6129 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:48.242830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8688b-1adc-4133-b6fb-0f9e2fb4eea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:48.244790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8688b-1adc-4133-b6fb-0f9e2fb4eea8 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:48.245576 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8688b-1adc-4133-b6fb-0f9e2fb4eea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:48.246230 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8688b-1adc-4133-b6fb-0f9e2fb4eea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:48.247929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8688b-1adc-4133-b6fb-0f9e2fb4eea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:48.250835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8688b-1adc-4133-b6fb-0f9e2fb4eea8 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:48.269405 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 944/1901] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:48.274085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb440c-8586-422c-bdf9-f0dc5893d0db admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:48.287755 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb440c-8586-422c-bdf9-f0dc5893d0db admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:48.296987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3183927-e059-4d2e-968e-7299ac55585e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:48.298627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3183927-e059-4d2e-968e-7299ac55585e admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:48.299043 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3183927-e059-4d2e-968e-7299ac55585e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:48.299474 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3183927-e059-4d2e-968e-7299ac55585e admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:48.300658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3183927-e059-4d2e-968e-7299ac55585e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:48.307846 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3183927-e059-4d2e-968e-7299ac55585e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:48.336188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 958/1902] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:48.342670 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8688b-1adc-4133-b6fb-0f9e2fb4eea8 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:48.358378 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8688b-1adc-4133-b6fb-0f9e2fb4eea8 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:48.371505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd9b16-0f86-4bfe-8271-c1b4f3c90d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:48.372792 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd9b16-0f86-4bfe-8271-c1b4f3c90d9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:48.373261 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd9b16-0f86-4bfe-8271-c1b4f3c90d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:48.373737 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd9b16-0f86-4bfe-8271-c1b4f3c90d9a admin admin] PATH_INFO: `/v3/projects/42a54966d5ec4e27890a4d699f6b6266/users/898aeff1647f4fc09408fa9221abb42f/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:48.415974 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd9b16-0f86-4bfe-8271-c1b4f3c90d9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a54966d5ec4e27890a4d699f6b6266, user_id=898aeff1647f4fc09408fa9221abb42f, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:48.429756 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd9b16-0f86-4bfe-8271-c1b4f3c90d9a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:48.483212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 945/1903] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:48 2022] PUT /identity/v3/projects/42a54966d5ec4e27890a4d699f6b6266/users/898aeff1647f4fc09408fa9221abb42f/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:48.503740 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b7f76-9800-47b9-8eea-223fe6e72c9c None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:48.504041 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b7f76-9800-47b9-8eea-223fe6e72c9c None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:48.504408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b7f76-9800-47b9-8eea-223fe6e72c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:48.571985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-da4b7f76-9800-47b9-8eea-223fe6e72c9c None None] MFA Rules not processed for user `898aeff1647f4fc09408fa9221abb42f`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:48.576305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-da4b7f76-9800-47b9-8eea-223fe6e72c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:48.670208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:48.689577 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 959/1904] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:48.694038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd9b16-0f86-4bfe-8271-c1b4f3c90d9a admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:48.826708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd9b16-0f86-4bfe-8271-c1b4f3c90d9a admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:48.834045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab619529-ffc0-468d-b23f-da68d875d52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:48.835278 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab619529-ffc0-468d-b23f-da68d875d52c admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:48.835794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab619529-ffc0-468d-b23f-da68d875d52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:48.845802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab619529-ffc0-468d-b23f-da68d875d52c admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:48.845802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab619529-ffc0-468d-b23f-da68d875d52c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:48.845802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab619529-ffc0-468d-b23f-da68d875d52c admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:48.992149 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 946/1905] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:48 2022] POST /identity/v3/projects => generated 373 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:32:48.995662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b7f76-9800-47b9-8eea-223fe6e72c9c None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:49.011021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b7f76-9800-47b9-8eea-223fe6e72c9c None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:49.096466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115c841-1699-4d10-9c28-9d95e981aad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:49.097250 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115c841-1699-4d10-9c28-9d95e981aad4 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:49.097548 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115c841-1699-4d10-9c28-9d95e981aad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:49.097856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115c841-1699-4d10-9c28-9d95e981aad4 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:49.098602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115c841-1699-4d10-9c28-9d95e981aad4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:49.102768 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115c841-1699-4d10-9c28-9d95e981aad4 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:49.268428 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 960/1906] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:48 2022] POST /identity/v3/users => generated 293 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:49.272921 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab619529-ffc0-468d-b23f-da68d875d52c admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:49.283593 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab619529-ffc0-468d-b23f-da68d875d52c admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:49.288996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab7694-11a6-4013-bd71-399338facf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:49.289765 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab7694-11a6-4013-bd71-399338facf1f admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:49.290084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab7694-11a6-4013-bd71-399338facf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:49.290602 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab7694-11a6-4013-bd71-399338facf1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:49.291598 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab7694-11a6-4013-bd71-399338facf1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:49.293288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab7694-11a6-4013-bd71-399338facf1f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:49.307162 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 947/1907] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:32:49.314963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115c841-1699-4d10-9c28-9d95e981aad4 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:49.336790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115c841-1699-4d10-9c28-9d95e981aad4 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:49.342911 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5d963-6cd5-4857-9e4e-02de72a9ef27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:49.343919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5d963-6cd5-4857-9e4e-02de72a9ef27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:49.344332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5d963-6cd5-4857-9e4e-02de72a9ef27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:49.344707 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5d963-6cd5-4857-9e4e-02de72a9ef27 admin admin] PATH_INFO: `/v3/projects/6ac9721d74214178bac8f53a94472dca/users/31bba208218f48f08dbb13d31ad6abaa/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:49.361831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5d963-6cd5-4857-9e4e-02de72a9ef27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ac9721d74214178bac8f53a94472dca, user_id=31bba208218f48f08dbb13d31ad6abaa, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:49.364516 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5d963-6cd5-4857-9e4e-02de72a9ef27 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:49.479874 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 961/1908] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:49 2022] PUT /identity/v3/projects/6ac9721d74214178bac8f53a94472dca/users/31bba208218f48f08dbb13d31ad6abaa/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:49.484291 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab7694-11a6-4013-bd71-399338facf1f admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:49.506179 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab7694-11a6-4013-bd71-399338facf1f admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:49.604671 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774fe0d-a029-4a33-b0c7-4dbc9cd9c72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:49.607040 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774fe0d-a029-4a33-b0c7-4dbc9cd9c72d admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:49.607562 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774fe0d-a029-4a33-b0c7-4dbc9cd9c72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:49.608286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774fe0d-a029-4a33-b0c7-4dbc9cd9c72d admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:49.609833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774fe0d-a029-4a33-b0c7-4dbc9cd9c72d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:49.615284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774fe0d-a029-4a33-b0c7-4dbc9cd9c72d admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:49.632145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 948/1909] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:49.637658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5d963-6cd5-4857-9e4e-02de72a9ef27 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:49.660558 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5d963-6cd5-4857-9e4e-02de72a9ef27 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:49.668534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8137b-7c72-46d8-8798-35f45a117a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:49.669626 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8137b-7c72-46d8-8798-35f45a117a16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:49.669951 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8137b-7c72-46d8-8798-35f45a117a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:49.670440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8137b-7c72-46d8-8798-35f45a117a16 admin admin] PATH_INFO: `/v3/projects/6ac9721d74214178bac8f53a94472dca/users/31bba208218f48f08dbb13d31ad6abaa/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:49.673050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8137b-7c72-46d8-8798-35f45a117a16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ac9721d74214178bac8f53a94472dca, user_id=31bba208218f48f08dbb13d31ad6abaa, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:49.676619 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8137b-7c72-46d8-8798-35f45a117a16 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:49.690537 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 962/1910] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:49 2022] PUT /identity/v3/projects/6ac9721d74214178bac8f53a94472dca/users/31bba208218f48f08dbb13d31ad6abaa/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:49.701698 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8b0d0-3f5f-4823-b187-56b6901247cb None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:49.702558 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8b0d0-3f5f-4823-b187-56b6901247cb None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:49.702996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8b0d0-3f5f-4823-b187-56b6901247cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:49.737873 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-b6e8b0d0-3f5f-4823-b187-56b6901247cb None None] MFA Rules not processed for user `31bba208218f48f08dbb13d31ad6abaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:49.739133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b6e8b0d0-3f5f-4823-b187-56b6901247cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:49.898009 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 949/1911] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:50.064024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8137b-7c72-46d8-8798-35f45a117a16 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:50.067709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-24b8137b-7c72-46d8-8798-35f45a117a16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:50.186080 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8137b-7c72-46d8-8798-35f45a117a16 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:50.190581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7752e-3ad5-4af7-a585-517123e1ceb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:50.191461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7752e-3ad5-4af7-a585-517123e1ceb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:50.191963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7752e-3ad5-4af7-a585-517123e1ceb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:50.192370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7752e-3ad5-4af7-a585-517123e1ceb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:50.294410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:50.317014 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7752e-3ad5-4af7-a585-517123e1ceb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:50.318638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7752e-3ad5-4af7-a585-517123e1ceb5 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:50.331005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 963/1912] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:51.424909 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:53.225842 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:53.506613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8b0d0-3f5f-4823-b187-56b6901247cb None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:53.509703 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b6e8b0d0-3f5f-4823-b187-56b6901247cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:53.589865 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:53.617519 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8b0d0-3f5f-4823-b187-56b6901247cb None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:53.626056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b671f-fc88-4f2d-9057-f58f31030c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:53.626056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b671f-fc88-4f2d-9057-f58f31030c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:53.626056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b671f-fc88-4f2d-9057-f58f31030c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:53.626056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b671f-fc88-4f2d-9057-f58f31030c99 admin admin] PATH_INFO: `/v3/users/336c572cd6154216a4ca6468fb6876fe` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:53.627568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b671f-fc88-4f2d-9057-f58f31030c99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=336c572cd6154216a4ca6468fb6876fe)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:53.629633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b671f-fc88-4f2d-9057-f58f31030c99 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:53.884378 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-a75b671f-fc88-4f2d-9057-f58f31030c99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '336c572cd6154216a4ca6468fb6876fe'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:53.981592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-a75b671f-fc88-4f2d-9057-f58f31030c99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '336c572cd6154216a4ca6468fb6876fe'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:53.981751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-a75b671f-fc88-4f2d-9057-f58f31030c99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '336c572cd6154216a4ca6468fb6876fe'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:54.000266 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-a75b671f-fc88-4f2d-9057-f58f31030c99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '336c572cd6154216a4ca6468fb6876fe'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:54.010178 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-a75b671f-fc88-4f2d-9057-f58f31030c99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '336c572cd6154216a4ca6468fb6876fe'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:54.022298 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 950/1913] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:53 2022] DELETE /identity/v3/users/336c572cd6154216a4ca6468fb6876fe => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:54.027969 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a516a0c-5c3b-48bc-9883-09579394edbc None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:54.028263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a516a0c-5c3b-48bc-9883-09579394edbc None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:54.028552 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a516a0c-5c3b-48bc-9883-09579394edbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:54.047431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6a516a0c-5c3b-48bc-9883-09579394edbc None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:54.048665 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6a516a0c-5c3b-48bc-9883-09579394edbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:54.125127 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b671f-fc88-4f2d-9057-f58f31030c99 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:54.126723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a75b671f-fc88-4f2d-9057-f58f31030c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:54.150467 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 964/1914] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:54.160066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a516a0c-5c3b-48bc-9883-09579394edbc None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:54.163280 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6a516a0c-5c3b-48bc-9883-09579394edbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:54.232490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b671f-fc88-4f2d-9057-f58f31030c99 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:54.240173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47af778-8541-4b36-a801-7a80df28c726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:54.241675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a516a0c-5c3b-48bc-9883-09579394edbc None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:54.245180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47af778-8541-4b36-a801-7a80df28c726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:54.245180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47af778-8541-4b36-a801-7a80df28c726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:54.245180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47af778-8541-4b36-a801-7a80df28c726 admin admin] PATH_INFO: `/v3/users/6482006517154d8b88824e3805c19403` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:54.246850 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47af778-8541-4b36-a801-7a80df28c726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6482006517154d8b88824e3805c19403)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:54.248947 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47af778-8541-4b36-a801-7a80df28c726 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:54.252345 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45941b17-d5ab-4664-8437-c333afed76a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:54.253349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45941b17-d5ab-4664-8437-c333afed76a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:54.260500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45941b17-d5ab-4664-8437-c333afed76a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:54.260500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45941b17-d5ab-4664-8437-c333afed76a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:54.269133 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45941b17-d5ab-4664-8437-c333afed76a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:54.271243 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45941b17-d5ab-4664-8437-c333afed76a1 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:54.283934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 965/1915] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:54.519151 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-b47af778-8541-4b36-a801-7a80df28c726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6482006517154d8b88824e3805c19403'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:54.664215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-b47af778-8541-4b36-a801-7a80df28c726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6482006517154d8b88824e3805c19403'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:54.674663 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-b47af778-8541-4b36-a801-7a80df28c726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6482006517154d8b88824e3805c19403'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:54.687225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-b47af778-8541-4b36-a801-7a80df28c726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6482006517154d8b88824e3805c19403'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:54.698088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:54.700776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-b47af778-8541-4b36-a801-7a80df28c726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6482006517154d8b88824e3805c19403'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:54.707622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 951/1916] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:54 2022] DELETE /identity/v3/users/6482006517154d8b88824e3805c19403 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:54.712769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87568c9-ff1e-4977-b84e-008cff84f165 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:54.713240 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87568c9-ff1e-4977-b84e-008cff84f165 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:54.713493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87568c9-ff1e-4977-b84e-008cff84f165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:54.733961 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-c87568c9-ff1e-4977-b84e-008cff84f165 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:54.735229 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c87568c9-ff1e-4977-b84e-008cff84f165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:54.798681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 966/1917] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:32:54.815398 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47af778-8541-4b36-a801-7a80df28c726 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:54.815398 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b47af778-8541-4b36-a801-7a80df28c726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:54.816161 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87568c9-ff1e-4977-b84e-008cff84f165 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:54.817707 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c87568c9-ff1e-4977-b84e-008cff84f165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:54.836173 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:54.848589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87568c9-ff1e-4977-b84e-008cff84f165 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:54.853452 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47909a94-bfbb-4212-8236-713dc431fb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:54.854405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47909a94-bfbb-4212-8236-713dc431fb0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:54.854693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47909a94-bfbb-4212-8236-713dc431fb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:54.854973 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47909a94-bfbb-4212-8236-713dc431fb0f admin admin] PATH_INFO: `/v3/projects/35e8192d825741039ebf0f25fc22511e` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:54.856256 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47909a94-bfbb-4212-8236-713dc431fb0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=35e8192d825741039ebf0f25fc22511e)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:54.862333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47909a94-bfbb-4212-8236-713dc431fb0f admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:54.901866 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47af778-8541-4b36-a801-7a80df28c726 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:54.906337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811ac0c-9f2f-4e6b-81e1-4ebfec7a87b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:54.907127 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811ac0c-9f2f-4e6b-81e1-4ebfec7a87b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:54.907540 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811ac0c-9f2f-4e6b-81e1-4ebfec7a87b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:54.907909 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811ac0c-9f2f-4e6b-81e1-4ebfec7a87b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:54.919659 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811ac0c-9f2f-4e6b-81e1-4ebfec7a87b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:54.921109 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811ac0c-9f2f-4e6b-81e1-4ebfec7a87b2 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:54.932016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 952/1918] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:55.020514 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-47909a94-bfbb-4212-8236-713dc431fb0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35e8192d825741039ebf0f25fc22511e', 'request_id': 'req-47909a94-bfbb-4212-8236-713dc431fb0f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.107306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-47909a94-bfbb-4212-8236-713dc431fb0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35e8192d825741039ebf0f25fc22511e', 'request_id': 'req-47909a94-bfbb-4212-8236-713dc431fb0f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.107404 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811ac0c-9f2f-4e6b-81e1-4ebfec7a87b2 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:55.107404 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8811ac0c-9f2f-4e6b-81e1-4ebfec7a87b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:55.107763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-47909a94-bfbb-4212-8236-713dc431fb0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35e8192d825741039ebf0f25fc22511e', 'request_id': 'req-47909a94-bfbb-4212-8236-713dc431fb0f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.120360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-47909a94-bfbb-4212-8236-713dc431fb0f admin admin] The token cache is being invalidate because project 35e8192d825741039ebf0f25fc22511e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:55.120547 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-47909a94-bfbb-4212-8236-713dc431fb0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.120810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811ac0c-9f2f-4e6b-81e1-4ebfec7a87b2 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:55.121160 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-47909a94-bfbb-4212-8236-713dc431fb0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.122909 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 967/1919] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:54 2022] DELETE /identity/v3/projects/35e8192d825741039ebf0f25fc22511e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:55.126743 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47909a94-bfbb-4212-8236-713dc431fb0f admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:55.128134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-47909a94-bfbb-4212-8236-713dc431fb0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:55.185273 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6448a4-7d88-4d09-8d83-6f07660f0e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:55.186422 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6448a4-7d88-4d09-8d83-6f07660f0e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:55.186602 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6448a4-7d88-4d09-8d83-6f07660f0e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:55.186840 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6448a4-7d88-4d09-8d83-6f07660f0e37 admin admin] PATH_INFO: `/v3/users/898aeff1647f4fc09408fa9221abb42f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:55.188168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6448a4-7d88-4d09-8d83-6f07660f0e37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=898aeff1647f4fc09408fa9221abb42f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:55.190449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6448a4-7d88-4d09-8d83-6f07660f0e37 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:55.212189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47909a94-bfbb-4212-8236-713dc431fb0f admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:55.219662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdef8a0-8537-4113-a3e0-dab0bbbdf482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:55.222857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdef8a0-8537-4113-a3e0-dab0bbbdf482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:55.222857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdef8a0-8537-4113-a3e0-dab0bbbdf482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:55.222857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdef8a0-8537-4113-a3e0-dab0bbbdf482 admin admin] PATH_INFO: `/v3/users/120dee5b704c424a836c0c9dfbab4368` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:55.227852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdef8a0-8537-4113-a3e0-dab0bbbdf482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=120dee5b704c424a836c0c9dfbab4368)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:55.230252 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdef8a0-8537-4113-a3e0-dab0bbbdf482 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:55.434104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-9d6448a4-7d88-4d09-8d83-6f07660f0e37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '898aeff1647f4fc09408fa9221abb42f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.466895 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-cbdef8a0-8537-4113-a3e0-dab0bbbdf482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '120dee5b704c424a836c0c9dfbab4368'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.477926 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:55.540907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-9d6448a4-7d88-4d09-8d83-6f07660f0e37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '898aeff1647f4fc09408fa9221abb42f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.541508 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-9d6448a4-7d88-4d09-8d83-6f07660f0e37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '898aeff1647f4fc09408fa9221abb42f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.557267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-9d6448a4-7d88-4d09-8d83-6f07660f0e37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '898aeff1647f4fc09408fa9221abb42f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.564633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-9d6448a4-7d88-4d09-8d83-6f07660f0e37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '898aeff1647f4fc09408fa9221abb42f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.571773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 953/1920] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:55 2022] DELETE /identity/v3/users/898aeff1647f4fc09408fa9221abb42f => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:55.575924 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6448a4-7d88-4d09-8d83-6f07660f0e37 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:55.576473 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-9d6448a4-7d88-4d09-8d83-6f07660f0e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:55.602931 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-cbdef8a0-8537-4113-a3e0-dab0bbbdf482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '120dee5b704c424a836c0c9dfbab4368'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.611436 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-cbdef8a0-8537-4113-a3e0-dab0bbbdf482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '120dee5b704c424a836c0c9dfbab4368'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.612282 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-cbdef8a0-8537-4113-a3e0-dab0bbbdf482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '120dee5b704c424a836c0c9dfbab4368'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.625813 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-cbdef8a0-8537-4113-a3e0-dab0bbbdf482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '120dee5b704c424a836c0c9dfbab4368'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.634126 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 968/1921] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:55 2022] DELETE /identity/v3/users/120dee5b704c424a836c0c9dfbab4368 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:55.636222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838abca2-8603-41ee-93ad-4d29ea7200e8 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:55.636222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838abca2-8603-41ee-93ad-4d29ea7200e8 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:55.636913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838abca2-8603-41ee-93ad-4d29ea7200e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:55.653952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6448a4-7d88-4d09-8d83-6f07660f0e37 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:55.657830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-838abca2-8603-41ee-93ad-4d29ea7200e8 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:55.659395 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-838abca2-8603-41ee-93ad-4d29ea7200e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:55.733351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:55.734637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:55.734925 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:55.736750 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc admin admin] PATH_INFO: `/v3/projects/b98c6f582c524955a474383d355a7612` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:55.738011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b98c6f582c524955a474383d355a7612)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:55.740684 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:55.744702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 969/1922] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:55.755935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838abca2-8603-41ee-93ad-4d29ea7200e8 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:55.757660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-838abca2-8603-41ee-93ad-4d29ea7200e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:55.820416 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838abca2-8603-41ee-93ad-4d29ea7200e8 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:55.824560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89309663-be59-4a2f-ada5-6fd7c47131bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:55.825215 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89309663-be59-4a2f-ada5-6fd7c47131bc service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:55.825510 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89309663-be59-4a2f-ada5-6fd7c47131bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:55.825786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89309663-be59-4a2f-ada5-6fd7c47131bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:55.835825 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89309663-be59-4a2f-ada5-6fd7c47131bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:55.837437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89309663-be59-4a2f-ada5-6fd7c47131bc service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:55.849019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 970/1923] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:32:55.900339 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b98c6f582c524955a474383d355a7612', 'request_id': 'req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.970572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b98c6f582c524955a474383d355a7612', 'request_id': 'req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.977188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b98c6f582c524955a474383d355a7612', 'request_id': 'req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.989677 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc admin admin] The token cache is being invalidate because project b98c6f582c524955a474383d355a7612 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:55.989969 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.990701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:55.995966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 954/1924] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:55 2022] DELETE /identity/v3/projects/b98c6f582c524955a474383d355a7612 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:55.999546 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89309663-be59-4a2f-ada5-6fd7c47131bc service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:56.001153 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-89309663-be59-4a2f-ada5-6fd7c47131bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:56.029555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89309663-be59-4a2f-ada5-6fd7c47131bc service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:56.116615 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939e647-bdeb-40f5-bdff-6bacb892b770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:56.117601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939e647-bdeb-40f5-bdff-6bacb892b770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:56.118436 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939e647-bdeb-40f5-bdff-6bacb892b770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:56.118714 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939e647-bdeb-40f5-bdff-6bacb892b770 admin admin] PATH_INFO: `/v3/users/4590129e58334bb1a02eeecd1be342c1` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:56.120747 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b939e647-bdeb-40f5-bdff-6bacb892b770 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4590129e58334bb1a02eeecd1be342c1)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:56.122768 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b939e647-bdeb-40f5-bdff-6bacb892b770 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:56.311713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:56.313394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:56.327823 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba1e1b-5d96-4f1d-aeac-d5c336889fbc admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:56.333277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eabd4c-3672-4964-8564-9862a787b8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:56.334578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eabd4c-3672-4964-8564-9862a787b8e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:56.335030 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eabd4c-3672-4964-8564-9862a787b8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:56.335580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eabd4c-3672-4964-8564-9862a787b8e3 admin admin] PATH_INFO: `/v3/projects/42b7a5e04a544e3a9836ce29c4bed110` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:56.337467 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eabd4c-3672-4964-8564-9862a787b8e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42b7a5e04a544e3a9836ce29c4bed110)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:56.341174 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eabd4c-3672-4964-8564-9862a787b8e3 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:56.405544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:56.520455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b939e647-bdeb-40f5-bdff-6bacb892b770 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4590129e58334bb1a02eeecd1be342c1'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:56.523513 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-83eabd4c-3672-4964-8564-9862a787b8e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42b7a5e04a544e3a9836ce29c4bed110', 'request_id': 'req-83eabd4c-3672-4964-8564-9862a787b8e3'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:56.666983 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b939e647-bdeb-40f5-bdff-6bacb892b770 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4590129e58334bb1a02eeecd1be342c1'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:56.677957 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b939e647-bdeb-40f5-bdff-6bacb892b770 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4590129e58334bb1a02eeecd1be342c1'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:56.679990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b939e647-bdeb-40f5-bdff-6bacb892b770 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4590129e58334bb1a02eeecd1be342c1'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:56.698892 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b939e647-bdeb-40f5-bdff-6bacb892b770 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4590129e58334bb1a02eeecd1be342c1'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:56.704999 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 971/1925] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:55 2022] DELETE /identity/v3/users/4590129e58334bb1a02eeecd1be342c1 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:56.705821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939e647-bdeb-40f5-bdff-6bacb892b770 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:56.707225 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b939e647-bdeb-40f5-bdff-6bacb892b770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:56.780015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-83eabd4c-3672-4964-8564-9862a787b8e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42b7a5e04a544e3a9836ce29c4bed110', 'request_id': 'req-83eabd4c-3672-4964-8564-9862a787b8e3'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:56.783076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:56.789008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-83eabd4c-3672-4964-8564-9862a787b8e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42b7a5e04a544e3a9836ce29c4bed110', 'request_id': 'req-83eabd4c-3672-4964-8564-9862a787b8e3'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:56.795637 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939e647-bdeb-40f5-bdff-6bacb892b770 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:56.803110 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858c5eb-4b6f-4d9a-9808-b67673c39f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:56.804647 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858c5eb-4b6f-4d9a-9808-b67673c39f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:56.805104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858c5eb-4b6f-4d9a-9808-b67673c39f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:56.805572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858c5eb-4b6f-4d9a-9808-b67673c39f45 admin admin] PATH_INFO: `/v3/projects/42a54966d5ec4e27890a4d699f6b6266` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:56.806448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-83eabd4c-3672-4964-8564-9862a787b8e3 admin admin] The token cache is being invalidate because project 42b7a5e04a544e3a9836ce29c4bed110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:56.806936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-83eabd4c-3672-4964-8564-9862a787b8e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:56.808163 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858c5eb-4b6f-4d9a-9808-b67673c39f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42a54966d5ec4e27890a4d699f6b6266)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:56.808335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-83eabd4c-3672-4964-8564-9862a787b8e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:56.810780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 955/1926] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:56 2022] DELETE /identity/v3/projects/42b7a5e04a544e3a9836ce29c4bed110 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:32:56.811657 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858c5eb-4b6f-4d9a-9808-b67673c39f45 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:56.815360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eabd4c-3672-4964-8564-9862a787b8e3 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:56.821474 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-83eabd4c-3672-4964-8564-9862a787b8e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:56.901672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eabd4c-3672-4964-8564-9862a787b8e3 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:56.908034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c12500-e203-4d79-9b1b-2803e3418650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:56.908881 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c12500-e203-4d79-9b1b-2803e3418650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:56.909208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c12500-e203-4d79-9b1b-2803e3418650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:56.909500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c12500-e203-4d79-9b1b-2803e3418650 admin admin] PATH_INFO: `/v3/users/0097a1e1de024ed0830ba5f7acb8c8e1` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:56.910835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c12500-e203-4d79-9b1b-2803e3418650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0097a1e1de024ed0830ba5f7acb8c8e1)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:56.912759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c12500-e203-4d79-9b1b-2803e3418650 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:56.994095 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7858c5eb-4b6f-4d9a-9808-b67673c39f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42a54966d5ec4e27890a4d699f6b6266', 'request_id': 'req-7858c5eb-4b6f-4d9a-9808-b67673c39f45'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.109305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7858c5eb-4b6f-4d9a-9808-b67673c39f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42a54966d5ec4e27890a4d699f6b6266', 'request_id': 'req-7858c5eb-4b6f-4d9a-9808-b67673c39f45'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.122187 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7858c5eb-4b6f-4d9a-9808-b67673c39f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42a54966d5ec4e27890a4d699f6b6266', 'request_id': 'req-7858c5eb-4b6f-4d9a-9808-b67673c39f45'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.136977 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7858c5eb-4b6f-4d9a-9808-b67673c39f45 admin admin] The token cache is being invalidate because project 42a54966d5ec4e27890a4d699f6b6266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:57.137264 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7858c5eb-4b6f-4d9a-9808-b67673c39f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.138168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7858c5eb-4b6f-4d9a-9808-b67673c39f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.140422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 972/1927] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:56 2022] DELETE /identity/v3/projects/42a54966d5ec4e27890a4d699f6b6266 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:57.142541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa84594-026f-4793-b94e-84c1110bed3d None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:57.142843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa84594-026f-4793-b94e-84c1110bed3d None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:57.143108 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa84594-026f-4793-b94e-84c1110bed3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:57.172602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-bfa84594-026f-4793-b94e-84c1110bed3d None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:57.173915 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-bfa84594-026f-4793-b94e-84c1110bed3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:57.194668 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-03c12500-e203-4d79-9b1b-2803e3418650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0097a1e1de024ed0830ba5f7acb8c8e1'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.244848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 973/1928] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:57.246296 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa84594-026f-4793-b94e-84c1110bed3d None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:57.248386 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-bfa84594-026f-4793-b94e-84c1110bed3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:57.263876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa84594-026f-4793-b94e-84c1110bed3d None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:57.268576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b1050-5d2a-4898-8827-f709df716498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:57.269762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b1050-5d2a-4898-8827-f709df716498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:57.270044 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b1050-5d2a-4898-8827-f709df716498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:57.270360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b1050-5d2a-4898-8827-f709df716498 admin admin] PATH_INFO: `/v3/users/31bba208218f48f08dbb13d31ad6abaa` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:57.271612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b1050-5d2a-4898-8827-f709df716498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31bba208218f48f08dbb13d31ad6abaa)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:57.273607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b1050-5d2a-4898-8827-f709df716498 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:57.320423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-03c12500-e203-4d79-9b1b-2803e3418650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0097a1e1de024ed0830ba5f7acb8c8e1'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.321081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-03c12500-e203-4d79-9b1b-2803e3418650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0097a1e1de024ed0830ba5f7acb8c8e1'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.335676 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-03c12500-e203-4d79-9b1b-2803e3418650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0097a1e1de024ed0830ba5f7acb8c8e1'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.345068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-03c12500-e203-4d79-9b1b-2803e3418650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0097a1e1de024ed0830ba5f7acb8c8e1'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.351679 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 956/1929] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:56 2022] DELETE /identity/v3/users/0097a1e1de024ed0830ba5f7acb8c8e1 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:57.353045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63596f-ec57-417f-aeec-ad137f1125d4 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:57.353339 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63596f-ec57-417f-aeec-ad137f1125d4 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:57.353624 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63596f-ec57-417f-aeec-ad137f1125d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:57.372463 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-cc63596f-ec57-417f-aeec-ad137f1125d4 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:57.373869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-cc63596f-ec57-417f-aeec-ad137f1125d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:57.437438 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 957/1930] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:57.442129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63596f-ec57-417f-aeec-ad137f1125d4 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:57.442129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-cc63596f-ec57-417f-aeec-ad137f1125d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:57.453073 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63596f-ec57-417f-aeec-ad137f1125d4 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:57.457888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:57.458802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:57.459083 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:57.459388 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe admin admin] PATH_INFO: `/v3/projects/f97c36ccb1054e4a9d9a2b90760c19ad` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:57.460601 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=f97c36ccb1054e4a9d9a2b90760c19ad)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:57.462702 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:57.510200 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-400b1050-5d2a-4898-8827-f709df716498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31bba208218f48f08dbb13d31ad6abaa'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.563546 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-400b1050-5d2a-4898-8827-f709df716498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31bba208218f48f08dbb13d31ad6abaa'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.572629 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f97c36ccb1054e4a9d9a2b90760c19ad', 'request_id': 'req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.574842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-400b1050-5d2a-4898-8827-f709df716498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31bba208218f48f08dbb13d31ad6abaa'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.575672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-400b1050-5d2a-4898-8827-f709df716498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31bba208218f48f08dbb13d31ad6abaa'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.589049 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-400b1050-5d2a-4898-8827-f709df716498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31bba208218f48f08dbb13d31ad6abaa'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.596309 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 974/1931] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:57 2022] DELETE /identity/v3/users/31bba208218f48f08dbb13d31ad6abaa => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:57.597638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbbe63-1d9d-495b-84bc-ac44ac2faf87 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:57.597919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbbe63-1d9d-495b-84bc-ac44ac2faf87 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:57.598390 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbbe63-1d9d-495b-84bc-ac44ac2faf87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:57.624907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-e6dbbe63-1d9d-495b-84bc-ac44ac2faf87 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:57.626292 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e6dbbe63-1d9d-495b-84bc-ac44ac2faf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:57.701559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 975/1932] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:57.702556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbbe63-1d9d-495b-84bc-ac44ac2faf87 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:57.703877 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e6dbbe63-1d9d-495b-84bc-ac44ac2faf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:57.712047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f97c36ccb1054e4a9d9a2b90760c19ad', 'request_id': 'req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.718153 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbbe63-1d9d-495b-84bc-ac44ac2faf87 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:57.724124 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c64ae5-c563-4e8c-adef-4739dd81b987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:57.724727 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c64ae5-c563-4e8c-adef-4739dd81b987 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:57.725005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c64ae5-c563-4e8c-adef-4739dd81b987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:57.725277 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c64ae5-c563-4e8c-adef-4739dd81b987 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:57.726063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c64ae5-c563-4e8c-adef-4739dd81b987 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:32:57.726406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c64ae5-c563-4e8c-adef-4739dd81b987 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:57.728270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c64ae5-c563-4e8c-adef-4739dd81b987 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:57.733330 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f97c36ccb1054e4a9d9a2b90760c19ad', 'request_id': 'req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.744393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 976/1933] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:32:57.744850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c64ae5-c563-4e8c-adef-4739dd81b987 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:57.747260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-58c64ae5-c563-4e8c-adef-4739dd81b987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:57.747727 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe admin admin] The token cache is being invalidate because project f97c36ccb1054e4a9d9a2b90760c19ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:57.748028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.748925 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:57.750870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 958/1934] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:57 2022] DELETE /identity/v3/projects/f97c36ccb1054e4a9d9a2b90760c19ad => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:57.751721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:57.753219 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:57.816013 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c64ae5-c563-4e8c-adef-4739dd81b987 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:57.822926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30e1ee-8acd-454b-804e-078992f89ab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:57.823491 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30e1ee-8acd-454b-804e-078992f89ab8 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:57.823767 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30e1ee-8acd-454b-804e-078992f89ab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:57.824044 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30e1ee-8acd-454b-804e-078992f89ab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:57.826046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ca30e1ee-8acd-454b-804e-078992f89ab8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:57.836457 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fdd61-f13e-4140-a7c7-6a70221b7fbe admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:57.838890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30e1ee-8acd-454b-804e-078992f89ab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:57.840897 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30e1ee-8acd-454b-804e-078992f89ab8 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:57.843776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829c0a6-e83f-4c85-9ba0-6e0cb9808c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:57.844827 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829c0a6-e83f-4c85-9ba0-6e0cb9808c86 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:57.845116 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829c0a6-e83f-4c85-9ba0-6e0cb9808c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:57.845401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829c0a6-e83f-4c85-9ba0-6e0cb9808c86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:57.847039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829c0a6-e83f-4c85-9ba0-6e0cb9808c86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:57.849900 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829c0a6-e83f-4c85-9ba0-6e0cb9808c86 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:57.852317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 977/1935] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:32:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:32:57.854062 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d77137-ffd2-4b99-983a-db1824bcdddb None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:57.854422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d77137-ffd2-4b99-983a-db1824bcdddb None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:57.854716 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d77137-ffd2-4b99-983a-db1824bcdddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:57.879791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-48d77137-ffd2-4b99-983a-db1824bcdddb None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:57.881028 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-48d77137-ffd2-4b99-983a-db1824bcdddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:57.896573 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 978/1936] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:57.908398 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b1e76-1dc9-4d2a-b97b-f24fff597ccb None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:57.908723 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b1e76-1dc9-4d2a-b97b-f24fff597ccb None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:57.909001 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b1e76-1dc9-4d2a-b97b-f24fff597ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:57.929607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-f41b1e76-1dc9-4d2a-b97b-f24fff597ccb None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:57.930799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f41b1e76-1dc9-4d2a-b97b-f24fff597ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:57.940034 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 979/1937] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:32:57.944117 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b1e76-1dc9-4d2a-b97b-f24fff597ccb None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:57.953767 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b1e76-1dc9-4d2a-b97b-f24fff597ccb None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:57.958786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa488281-2cb3-43de-a4bc-c619d8d9975e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:57.959478 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa488281-2cb3-43de-a4bc-c619d8d9975e admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:57.959767 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa488281-2cb3-43de-a4bc-c619d8d9975e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:57.960043 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa488281-2cb3-43de-a4bc-c619d8d9975e admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:57.960791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa488281-2cb3-43de-a4bc-c619d8d9975e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:32:57.961104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa488281-2cb3-43de-a4bc-c619d8d9975e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:57.962846 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa488281-2cb3-43de-a4bc-c619d8d9975e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:57.980408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 980/1938] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:32:57.980966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 959/1939] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:57 2022] POST /identity/v3/projects => generated 365 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:57.983199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa488281-2cb3-43de-a4bc-c619d8d9975e admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:57.983792 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829c0a6-e83f-4c85-9ba0-6e0cb9808c86 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:57.995174 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa488281-2cb3-43de-a4bc-c619d8d9975e admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:58.063167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:58.071037 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829c0a6-e83f-4c85-9ba0-6e0cb9808c86 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:58.075484 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4176251-74f5-4499-a692-435c0792311f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:58.076635 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4176251-74f5-4499-a692-435c0792311f admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:58.077147 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4176251-74f5-4499-a692-435c0792311f admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:58.077607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4176251-74f5-4499-a692-435c0792311f admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:58.077762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364b189-7828-43d9-9b28-067917687f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:58.078653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364b189-7828-43d9-9b28-067917687f53 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:58.078937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4176251-74f5-4499-a692-435c0792311f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:58.079334 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364b189-7828-43d9-9b28-067917687f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:58.079710 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364b189-7828-43d9-9b28-067917687f53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:58.080570 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364b189-7828-43d9-9b28-067917687f53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:58.082491 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4176251-74f5-4499-a692-435c0792311f admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:58.082835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364b189-7828-43d9-9b28-067917687f53 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:58.223415 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 981/1940] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:57 2022] POST /identity/v3/users => generated 283 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:58.229049 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4176251-74f5-4499-a692-435c0792311f admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:58.240774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4176251-74f5-4499-a692-435c0792311f admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:58.248525 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3868d-ca47-4e6d-a07f-dd06de882c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:58.248802 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3868d-ca47-4e6d-a07f-dd06de882c08 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:58.253160 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3868d-ca47-4e6d-a07f-dd06de882c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:58.253672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3868d-ca47-4e6d-a07f-dd06de882c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:58.254992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d3868d-ca47-4e6d-a07f-dd06de882c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:58.257823 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d3868d-ca47-4e6d-a07f-dd06de882c08 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:58.263604 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 960/1941] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:57 2022] POST /identity/v3/projects => generated 387 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:58.264931 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364b189-7828-43d9-9b28-067917687f53 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:58.269507 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5364b189-7828-43d9-9b28-067917687f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:58.279896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 982/1942] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:32:58.281835 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3868d-ca47-4e6d-a07f-dd06de882c08 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:58.299590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3868d-ca47-4e6d-a07f-dd06de882c08 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:58.380679 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364b189-7828-43d9-9b28-067917687f53 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:58.388240 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a88914-8e9e-4033-bc3c-8809cd400d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:58.389227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a88914-8e9e-4033-bc3c-8809cd400d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:58.389515 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a88914-8e9e-4033-bc3c-8809cd400d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:58.389788 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a88914-8e9e-4033-bc3c-8809cd400d2b admin admin] PATH_INFO: `/v3/projects/6ac9721d74214178bac8f53a94472dca` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:58.392169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a88914-8e9e-4033-bc3c-8809cd400d2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ac9721d74214178bac8f53a94472dca)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:58.393532 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a88914-8e9e-4033-bc3c-8809cd400d2b admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:58.438700 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676d547-9be4-4742-a464-7f4a59c8051e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:58.449360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676d547-9be4-4742-a464-7f4a59c8051e admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:58.449360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676d547-9be4-4742-a464-7f4a59c8051e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:58.449360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676d547-9be4-4742-a464-7f4a59c8051e admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:58.449360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676d547-9be4-4742-a464-7f4a59c8051e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:58.449360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676d547-9be4-4742-a464-7f4a59c8051e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:58.640527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-68a88914-8e9e-4033-bc3c-8809cd400d2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ac9721d74214178bac8f53a94472dca', 'request_id': 'req-68a88914-8e9e-4033-bc3c-8809cd400d2b'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:58.647117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:58.655529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 983/1943] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:58 2022] POST /identity/v3/users => generated 294 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:58.656485 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676d547-9be4-4742-a464-7f4a59c8051e admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:58.671560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676d547-9be4-4742-a464-7f4a59c8051e admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:58.752922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1cae8-7f36-4c8c-86c4-2f316d924bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:58.756063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1cae8-7f36-4c8c-86c4-2f316d924bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:58.756063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1cae8-7f36-4c8c-86c4-2f316d924bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:58.756063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1cae8-7f36-4c8c-86c4-2f316d924bd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:58.756063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef1cae8-7f36-4c8c-86c4-2f316d924bd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:58.756685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef1cae8-7f36-4c8c-86c4-2f316d924bd1 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:58.770506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 984/1944] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:58.771608 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1cae8-7f36-4c8c-86c4-2f316d924bd1 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:58.773738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-fef1cae8-7f36-4c8c-86c4-2f316d924bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:58.786425 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1cae8-7f36-4c8c-86c4-2f316d924bd1 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:58.796054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6d04c-38af-459c-b46a-dbe316daeed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:58.796054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6d04c-38af-459c-b46a-dbe316daeed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:58.796054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6d04c-38af-459c-b46a-dbe316daeed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:58.796054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6d04c-38af-459c-b46a-dbe316daeed3 admin admin] PATH_INFO: `/v3/projects/7f89beb8dc8d4f3ba14717e14c527d7b` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:58.796607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab6d04c-38af-459c-b46a-dbe316daeed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f89beb8dc8d4f3ba14717e14c527d7b)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:58.796607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab6d04c-38af-459c-b46a-dbe316daeed3 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:58.819913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-68a88914-8e9e-4033-bc3c-8809cd400d2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ac9721d74214178bac8f53a94472dca', 'request_id': 'req-68a88914-8e9e-4033-bc3c-8809cd400d2b'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:58.828544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-68a88914-8e9e-4033-bc3c-8809cd400d2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ac9721d74214178bac8f53a94472dca', 'request_id': 'req-68a88914-8e9e-4033-bc3c-8809cd400d2b'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:58.841174 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-68a88914-8e9e-4033-bc3c-8809cd400d2b admin admin] The token cache is being invalidate because project 6ac9721d74214178bac8f53a94472dca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:58.841686 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-68a88914-8e9e-4033-bc3c-8809cd400d2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:58.842522 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-68a88914-8e9e-4033-bc3c-8809cd400d2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:58.844164 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 961/1945] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:58 2022] DELETE /identity/v3/projects/6ac9721d74214178bac8f53a94472dca => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:58.846387 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a88914-8e9e-4033-bc3c-8809cd400d2b admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:58.846625 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-68a88914-8e9e-4033-bc3c-8809cd400d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:58.860205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a88914-8e9e-4033-bc3c-8809cd400d2b admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:58.865087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ebf9d-d919-4dab-92c5-350464d8323b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:58.865883 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ebf9d-d919-4dab-92c5-350464d8323b admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:58.866168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ebf9d-d919-4dab-92c5-350464d8323b admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:58.866485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ebf9d-d919-4dab-92c5-350464d8323b admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:58.870437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ebf9d-d919-4dab-92c5-350464d8323b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:58.870437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ebf9d-d919-4dab-92c5-350464d8323b admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:58.882394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 962/1946] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:58.883386 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ebf9d-d919-4dab-92c5-350464d8323b admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:58.885618 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0d6ebf9d-d919-4dab-92c5-350464d8323b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:58.903672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ebf9d-d919-4dab-92c5-350464d8323b admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:58.913817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-9ab6d04c-38af-459c-b46a-dbe316daeed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f89beb8dc8d4f3ba14717e14c527d7b', 'request_id': 'req-9ab6d04c-38af-459c-b46a-dbe316daeed3'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:58.969031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee24b15-4659-4d72-8a9e-eedcf11ace55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:58.969946 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee24b15-4659-4d72-8a9e-eedcf11ace55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:58.970233 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee24b15-4659-4d72-8a9e-eedcf11ace55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:58.970548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee24b15-4659-4d72-8a9e-eedcf11ace55 admin admin] PATH_INFO: `/v3/projects/a85f65025aac4a20adcee358175d0aa1/users/f001c05d64d04c69b28468b38fa98916/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:58.990527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee24b15-4659-4d72-8a9e-eedcf11ace55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a85f65025aac4a20adcee358175d0aa1, user_id=f001c05d64d04c69b28468b38fa98916, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:58.993723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee24b15-4659-4d72-8a9e-eedcf11ace55 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:59.002976 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:59.071389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-9ab6d04c-38af-459c-b46a-dbe316daeed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f89beb8dc8d4f3ba14717e14c527d7b', 'request_id': 'req-9ab6d04c-38af-459c-b46a-dbe316daeed3'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:59.080755 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-9ab6d04c-38af-459c-b46a-dbe316daeed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f89beb8dc8d4f3ba14717e14c527d7b', 'request_id': 'req-9ab6d04c-38af-459c-b46a-dbe316daeed3'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:59.094178 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-9ab6d04c-38af-459c-b46a-dbe316daeed3 admin admin] The token cache is being invalidate because project 7f89beb8dc8d4f3ba14717e14c527d7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:32:59.094178 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-9ab6d04c-38af-459c-b46a-dbe316daeed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:59.094611 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-9ab6d04c-38af-459c-b46a-dbe316daeed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:59.101613 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 985/1947] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:32:58 2022] DELETE /identity/v3/projects/7f89beb8dc8d4f3ba14717e14c527d7b => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:59.101613 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e372cee-e961-4bb8-8b84-fc03b4303618 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:59.101613 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e372cee-e961-4bb8-8b84-fc03b4303618 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:59.101613 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e372cee-e961-4bb8-8b84-fc03b4303618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:59.118870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 963/1948] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:58 2022] PUT /identity/v3/projects/a85f65025aac4a20adcee358175d0aa1/users/f001c05d64d04c69b28468b38fa98916/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:59.119194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-7e372cee-e961-4bb8-8b84-fc03b4303618 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:59.120514 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-7e372cee-e961-4bb8-8b84-fc03b4303618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:59.127755 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee24b15-4659-4d72-8a9e-eedcf11ace55 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:59.130481 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-aee24b15-4659-4d72-8a9e-eedcf11ace55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:59.202509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 986/1949] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:59.203541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e372cee-e961-4bb8-8b84-fc03b4303618 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:59.205552 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-7e372cee-e961-4bb8-8b84-fc03b4303618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:59.222799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e372cee-e961-4bb8-8b84-fc03b4303618 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:59.224322 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee24b15-4659-4d72-8a9e-eedcf11ace55 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:59.232026 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb875f-cf3b-484a-a8bd-437205361961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:59.233139 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb875f-cf3b-484a-a8bd-437205361961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:59.233388 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ab6a8-3bf2-4d73-9d0e-23c9f057c675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:59.233563 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb875f-cf3b-484a-a8bd-437205361961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:59.233827 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb875f-cf3b-484a-a8bd-437205361961 admin admin] PATH_INFO: `/v3/users/b48c89db5c56462a8341b049b4a7108f` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:59.234224 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ab6a8-3bf2-4d73-9d0e-23c9f057c675 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:59.234691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ab6a8-3bf2-4d73-9d0e-23c9f057c675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:59.235039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ab6a8-3bf2-4d73-9d0e-23c9f057c675 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:59.235794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb875f-cf3b-484a-a8bd-437205361961 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b48c89db5c56462a8341b049b4a7108f)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:59.235979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ab6a8-3bf2-4d73-9d0e-23c9f057c675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:59.238188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ab6a8-3bf2-4d73-9d0e-23c9f057c675 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:59.238530 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb875f-cf3b-484a-a8bd-437205361961 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:59.255951 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 964/1950] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:32:59.257660 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51522029-841e-407a-8c47-265dd15c14e0 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:59.257960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51522029-841e-407a-8c47-265dd15c14e0 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:59.258243 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51522029-841e-407a-8c47-265dd15c14e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:59.292426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-51522029-841e-407a-8c47-265dd15c14e0 None None] MFA Rules not processed for user `f001c05d64d04c69b28468b38fa98916`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:59.293845 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-51522029-841e-407a-8c47-265dd15c14e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:59.384098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 965/1951] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:59.385646 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e7608-0b90-4a18-be2c-62b9ef144f9f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:59.385948 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e7608-0b90-4a18-be2c-62b9ef144f9f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:59.386236 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e7608-0b90-4a18-be2c-62b9ef144f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:59.406152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-ba2e7608-0b90-4a18-be2c-62b9ef144f9f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:59.407376 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ba2e7608-0b90-4a18-be2c-62b9ef144f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:59.420477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 966/1952] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:32:59.421635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e7608-0b90-4a18-be2c-62b9ef144f9f None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:59.437211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e7608-0b90-4a18-be2c-62b9ef144f9f None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:59.445440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d4b13-f9fc-4cb2-a792-e3f0b9442872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:59.447184 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d4b13-f9fc-4cb2-a792-e3f0b9442872 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:59.447674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d4b13-f9fc-4cb2-a792-e3f0b9442872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:59.448178 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d4b13-f9fc-4cb2-a792-e3f0b9442872 admin admin] PATH_INFO: `/v3/projects/47512a1c991a48cf96d492a4b25eec9f/users/ea27f45302e94a30894669344b028e13/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:59.480333 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d4b13-f9fc-4cb2-a792-e3f0b9442872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47512a1c991a48cf96d492a4b25eec9f, user_id=ea27f45302e94a30894669344b028e13, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:59.485007 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d4b13-f9fc-4cb2-a792-e3f0b9442872 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:59.544523 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-16cb875f-cf3b-484a-a8bd-437205361961 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b48c89db5c56462a8341b049b4a7108f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:59.640103 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 967/1953] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:32:59 2022] PUT /identity/v3/projects/47512a1c991a48cf96d492a4b25eec9f/users/ea27f45302e94a30894669344b028e13/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:59.641333 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d4b13-f9fc-4cb2-a792-e3f0b9442872 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:59.644202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f38d4b13-f9fc-4cb2-a792-e3f0b9442872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:59.665792 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-16cb875f-cf3b-484a-a8bd-437205361961 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b48c89db5c56462a8341b049b4a7108f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:59.674771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-16cb875f-cf3b-484a-a8bd-437205361961 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b48c89db5c56462a8341b049b4a7108f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:59.676233 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-16cb875f-cf3b-484a-a8bd-437205361961 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b48c89db5c56462a8341b049b4a7108f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:59.701241 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-16cb875f-cf3b-484a-a8bd-437205361961 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b48c89db5c56462a8341b049b4a7108f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:32:59.713083 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 987/1954] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:32:59 2022] DELETE /identity/v3/users/b48c89db5c56462a8341b049b4a7108f => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:32:59.716193 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb875f-cf3b-484a-a8bd-437205361961 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:32:59.727847 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d4b13-f9fc-4cb2-a792-e3f0b9442872 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:59.729833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-16cb875f-cf3b-484a-a8bd-437205361961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:59.826376 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f143e-5117-40f3-9811-40121390e980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:59.827352 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f143e-5117-40f3-9811-40121390e980 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:59.828312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f143e-5117-40f3-9811-40121390e980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:59.828312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f143e-5117-40f3-9811-40121390e980 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:59.829047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f143e-5117-40f3-9811-40121390e980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:59.831182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f143e-5117-40f3-9811-40121390e980 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:59.839548 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb875f-cf3b-484a-a8bd-437205361961 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:32:59.846175 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d75d8-a5d5-46d5-b1b4-801798b66d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:32:59.847063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d75d8-a5d5-46d5-b1b4-801798b66d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:59.847343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d75d8-a5d5-46d5-b1b4-801798b66d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:59.847626 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d75d8-a5d5-46d5-b1b4-801798b66d9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:59.848595 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d75d8-a5d5-46d5-b1b4-801798b66d9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:32:59.848908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d75d8-a5d5-46d5-b1b4-801798b66d9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:32:59.850861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d75d8-a5d5-46d5-b1b4-801798b66d9d admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:32:59.866887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 988/1955] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:32:59.870013 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e5b58-5649-4cf5-9b4a-f9c49c0d49f3 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:32:59.870515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e5b58-5649-4cf5-9b4a-f9c49c0d49f3 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:32:59.871040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e5b58-5649-4cf5-9b4a-f9c49c0d49f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:32:59.913368 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-bc1e5b58-5649-4cf5-9b4a-f9c49c0d49f3 None None] MFA Rules not processed for user `ea27f45302e94a30894669344b028e13`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:32:59.914729 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-bc1e5b58-5649-4cf5-9b4a-f9c49c0d49f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:32:59.918851 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:32:59.986421 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 968/1956] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:59 2022] POST /identity/v3/projects => generated 363 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:32:59.986421 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f143e-5117-40f3-9811-40121390e980 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:00.008076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 989/1957] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:32:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:00.008076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e5b58-5649-4cf5-9b4a-f9c49c0d49f3 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:00.008076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-bc1e5b58-5649-4cf5-9b4a-f9c49c0d49f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:00.020834 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e5b58-5649-4cf5-9b4a-f9c49c0d49f3 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:00.074612 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f143e-5117-40f3-9811-40121390e980 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:00.079861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d45b4-4054-43ef-8486-1288658b99f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:00.080687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d45b4-4054-43ef-8486-1288658b99f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:00.080896 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d45b4-4054-43ef-8486-1288658b99f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:00.081238 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d45b4-4054-43ef-8486-1288658b99f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:00.081998 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d45b4-4054-43ef-8486-1288658b99f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:00.084367 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d45b4-4054-43ef-8486-1288658b99f4 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:00.121449 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0b5c7-f89f-4794-a660-38796f639b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:00.122600 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0b5c7-f89f-4794-a660-38796f639b88 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:00.123071 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0b5c7-f89f-4794-a660-38796f639b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:00.123504 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0b5c7-f89f-4794-a660-38796f639b88 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:00.124736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0b5c7-f89f-4794-a660-38796f639b88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:00.128202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0b5c7-f89f-4794-a660-38796f639b88 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:00.168110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:00.215835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 969/1958] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:32:59 2022] POST /identity/v3/projects => generated 367 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:00.216754 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d45b4-4054-43ef-8486-1288658b99f4 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:00.218653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e81d45b4-4054-43ef-8486-1288658b99f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:00.283576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 990/1959] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:32:59 2022] POST /identity/v3/users => generated 288 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:00.284324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0b5c7-f89f-4794-a660-38796f639b88 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:00.297792 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d45b4-4054-43ef-8486-1288658b99f4 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:00.300858 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0b5c7-f89f-4794-a660-38796f639b88 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:00.303128 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad4419-5087-41a6-bcdc-cd11a22af324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:00.303809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad4419-5087-41a6-bcdc-cd11a22af324 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:00.304179 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad4419-5087-41a6-bcdc-cd11a22af324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:00.304462 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad4419-5087-41a6-bcdc-cd11a22af324 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:00.305204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad4419-5087-41a6-bcdc-cd11a22af324 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:00.307821 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad4419-5087-41a6-bcdc-cd11a22af324 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:00.310006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fe60b-6df2-4a4f-b8f1-2b535541532e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:00.311165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fe60b-6df2-4a4f-b8f1-2b535541532e admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:00.311591 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fe60b-6df2-4a4f-b8f1-2b535541532e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:00.312221 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fe60b-6df2-4a4f-b8f1-2b535541532e admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:00.313416 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fe60b-6df2-4a4f-b8f1-2b535541532e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:00.318294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fe60b-6df2-4a4f-b8f1-2b535541532e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:00.483562 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 991/1960] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:33:00 2022] POST /identity/v3/users => generated 284 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:00.484338 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fe60b-6df2-4a4f-b8f1-2b535541532e admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:00.489618 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 970/1961] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:33:00 2022] POST /identity/v3/projects => generated 383 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:00.490547 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad4419-5087-41a6-bcdc-cd11a22af324 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:00.495148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-66ad4419-5087-41a6-bcdc-cd11a22af324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:00.498633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fe60b-6df2-4a4f-b8f1-2b535541532e admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:00.553536 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:00.567440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1612db-7ca6-4b1c-9f20-bfb059516002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:00.568274 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1612db-7ca6-4b1c-9f20-bfb059516002 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:00.568558 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1612db-7ca6-4b1c-9f20-bfb059516002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:00.568893 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1612db-7ca6-4b1c-9f20-bfb059516002 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:00.569628 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1612db-7ca6-4b1c-9f20-bfb059516002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:00.570022 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad4419-5087-41a6-bcdc-cd11a22af324 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:00.571295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1612db-7ca6-4b1c-9f20-bfb059516002 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:00.574945 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652e229-d65b-4658-a0eb-11944a56d62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:00.575819 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652e229-d65b-4658-a0eb-11944a56d62a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:00.576108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652e229-d65b-4658-a0eb-11944a56d62a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:00.576383 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652e229-d65b-4658-a0eb-11944a56d62a admin admin] PATH_INFO: `/v3/projects/4a1d4519396f464f9be17dfe76d75aab` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:00.577651 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652e229-d65b-4658-a0eb-11944a56d62a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a1d4519396f464f9be17dfe76d75aab)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:00.579549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652e229-d65b-4658-a0eb-11944a56d62a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:00.588547 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 992/1962] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:00.590777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1612db-7ca6-4b1c-9f20-bfb059516002 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:00.611520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1612db-7ca6-4b1c-9f20-bfb059516002 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:00.619690 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5edad-3376-40b1-b723-d096a8e410d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:00.620532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5edad-3376-40b1-b723-d096a8e410d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:00.620820 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5edad-3376-40b1-b723-d096a8e410d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:00.621107 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5edad-3376-40b1-b723-d096a8e410d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:00.622339 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a5edad-3376-40b1-b723-d096a8e410d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:00.625164 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a5edad-3376-40b1-b723-d096a8e410d1 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:00.640737 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 993/1963] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:00.641775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5edad-3376-40b1-b723-d096a8e410d1 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:00.653941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5edad-3376-40b1-b723-d096a8e410d1 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:00.659199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8e84e-973f-4e22-a9c9-bf69a08a2957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:00.659891 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8e84e-973f-4e22-a9c9-bf69a08a2957 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:00.660221 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8e84e-973f-4e22-a9c9-bf69a08a2957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:00.660521 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8e84e-973f-4e22-a9c9-bf69a08a2957 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:00.661214 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8e84e-973f-4e22-a9c9-bf69a08a2957 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:00.663256 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8e84e-973f-4e22-a9c9-bf69a08a2957 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:00.726879 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5652e229-d65b-4658-a0eb-11944a56d62a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a1d4519396f464f9be17dfe76d75aab', 'request_id': 'req-5652e229-d65b-4658-a0eb-11944a56d62a'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:00.769424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 994/1964] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:33:00 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:00.770667 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8e84e-973f-4e22-a9c9-bf69a08a2957 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:00.788582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8e84e-973f-4e22-a9c9-bf69a08a2957 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:00.837624 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5652e229-d65b-4658-a0eb-11944a56d62a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a1d4519396f464f9be17dfe76d75aab', 'request_id': 'req-5652e229-d65b-4658-a0eb-11944a56d62a'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:00.847612 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5652e229-d65b-4658-a0eb-11944a56d62a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a1d4519396f464f9be17dfe76d75aab', 'request_id': 'req-5652e229-d65b-4658-a0eb-11944a56d62a'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:00.859263 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5652e229-d65b-4658-a0eb-11944a56d62a admin admin] The token cache is being invalidate because project 4a1d4519396f464f9be17dfe76d75aab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:33:00.859548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5652e229-d65b-4658-a0eb-11944a56d62a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:00.860232 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5652e229-d65b-4658-a0eb-11944a56d62a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:00.861836 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 971/1965] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:33:00 2022] DELETE /identity/v3/projects/4a1d4519396f464f9be17dfe76d75aab => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:00.862633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652e229-d65b-4658-a0eb-11944a56d62a admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:00.862930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de0d25-8beb-4e17-b2e8-c8015f48737b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:00.863972 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5652e229-d65b-4658-a0eb-11944a56d62a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:00.864889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de0d25-8beb-4e17-b2e8-c8015f48737b admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:00.865625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de0d25-8beb-4e17-b2e8-c8015f48737b admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:00.866811 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de0d25-8beb-4e17-b2e8-c8015f48737b admin admin] PATH_INFO: `/v3/projects/0050d55d0fad4d90a10e12132f7f69ee/users/3b913d89578c4ccb8a2257057b88ca78/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:00.881101 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652e229-d65b-4658-a0eb-11944a56d62a admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:00.890481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de0d25-8beb-4e17-b2e8-c8015f48737b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0050d55d0fad4d90a10e12132f7f69ee, user_id=3b913d89578c4ccb8a2257057b88ca78, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:00.891108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b222459-941c-408d-8393-f326afc43a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:00.892167 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b222459-941c-408d-8393-f326afc43a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:00.892637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b222459-941c-408d-8393-f326afc43a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:00.893104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b222459-941c-408d-8393-f326afc43a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:00.894412 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b222459-941c-408d-8393-f326afc43a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:00.895758 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de0d25-8beb-4e17-b2e8-c8015f48737b admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:00.897218 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b222459-941c-408d-8393-f326afc43a67 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:00.910592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 972/1966] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:00.911537 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b222459-941c-408d-8393-f326afc43a67 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:00.913477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0b222459-941c-408d-8393-f326afc43a67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:00.928391 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b222459-941c-408d-8393-f326afc43a67 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:00.933313 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230196f-31cc-4efe-bd9b-75874611bcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:00.933992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230196f-31cc-4efe-bd9b-75874611bcf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:00.934306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230196f-31cc-4efe-bd9b-75874611bcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:00.934596 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230196f-31cc-4efe-bd9b-75874611bcf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:00.935333 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230196f-31cc-4efe-bd9b-75874611bcf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:00.936939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230196f-31cc-4efe-bd9b-75874611bcf8 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:00.948085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 973/1967] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:00.949596 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192e2f4-9274-4687-87c6-153ce1dc1ecb None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:00.949885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192e2f4-9274-4687-87c6-153ce1dc1ecb None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:00.950168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192e2f4-9274-4687-87c6-153ce1dc1ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:00.970097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-9192e2f4-9274-4687-87c6-153ce1dc1ecb None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:00.971600 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-9192e2f4-9274-4687-87c6-153ce1dc1ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:00.980544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 974/1968] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:00.981505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192e2f4-9274-4687-87c6-153ce1dc1ecb None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:00.991631 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192e2f4-9274-4687-87c6-153ce1dc1ecb None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:00.997147 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8df69-bbf7-4d13-bf1e-7a7ba4dd45c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:00.998645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8df69-bbf7-4d13-bf1e-7a7ba4dd45c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:00.998645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8df69-bbf7-4d13-bf1e-7a7ba4dd45c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:00.998645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8df69-bbf7-4d13-bf1e-7a7ba4dd45c9 admin admin] PATH_INFO: `/v3/projects/c2759635a13e44ed9518b88baa879471/users/d5c945c9e9874c14b53c212c1d836e11/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:01.016513 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8df69-bbf7-4d13-bf1e-7a7ba4dd45c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2759635a13e44ed9518b88baa879471, user_id=d5c945c9e9874c14b53c212c1d836e11, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:01.019352 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8df69-bbf7-4d13-bf1e-7a7ba4dd45c9 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:01.029255 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 995/1969] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:00 2022] PUT /identity/v3/projects/0050d55d0fad4d90a10e12132f7f69ee/users/3b913d89578c4ccb8a2257057b88ca78/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:01.033732 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de0d25-8beb-4e17-b2e8-c8015f48737b admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:01.058359 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:01.107198 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de0d25-8beb-4e17-b2e8-c8015f48737b admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:01.112492 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9e0e6-820b-4284-b7db-66b8b7723df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:01.113215 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9e0e6-820b-4284-b7db-66b8b7723df4 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:01.114190 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9e0e6-820b-4284-b7db-66b8b7723df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:01.114495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9e0e6-820b-4284-b7db-66b8b7723df4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:01.115302 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9e0e6-820b-4284-b7db-66b8b7723df4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:01.129753 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9e0e6-820b-4284-b7db-66b8b7723df4 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:01.129918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 975/1970] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:00 2022] PUT /identity/v3/projects/c2759635a13e44ed9518b88baa879471/users/d5c945c9e9874c14b53c212c1d836e11/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:01.129918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6e0cd-525c-450d-898d-03034ec62e75 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:01.129918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6e0cd-525c-450d-898d-03034ec62e75 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:01.129918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6e0cd-525c-450d-898d-03034ec62e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:01.131556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 996/1971] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:01.132634 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9e0e6-820b-4284-b7db-66b8b7723df4 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:01.134882 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d1d9e0e6-820b-4284-b7db-66b8b7723df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:01.147801 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9e0e6-820b-4284-b7db-66b8b7723df4 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:01.152613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:01.158670 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-e4a6e0cd-525c-450d-898d-03034ec62e75 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:01.159917 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e4a6e0cd-525c-450d-898d-03034ec62e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:01.213576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770dd1bb-b503-47a1-bf3f-57cf605b8be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:01.214320 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dd1bb-b503-47a1-bf3f-57cf605b8be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:01.214612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dd1bb-b503-47a1-bf3f-57cf605b8be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:01.215146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dd1bb-b503-47a1-bf3f-57cf605b8be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:01.215979 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770dd1bb-b503-47a1-bf3f-57cf605b8be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:01.217604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770dd1bb-b503-47a1-bf3f-57cf605b8be5 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:01.228248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 997/1972] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:01.229758 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:01.230212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 976/1973] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:01.232150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6e0cd-525c-450d-898d-03034ec62e75 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:01.234749 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bcca8-06d1-4cb5-b9e1-208636f2ac49 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:01.238116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bcca8-06d1-4cb5-b9e1-208636f2ac49 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:01.238116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bcca8-06d1-4cb5-b9e1-208636f2ac49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:01.242429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6e0cd-525c-450d-898d-03034ec62e75 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:01.248205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f37cc0-3e09-47df-8b43-f83c96c32fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:01.249255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f37cc0-3e09-47df-8b43-f83c96c32fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:01.249544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f37cc0-3e09-47df-8b43-f83c96c32fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:01.249823 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f37cc0-3e09-47df-8b43-f83c96c32fb1 admin admin] PATH_INFO: `/v3/projects/00338469c4d942389e2c5767bbdaee18/users/0ffdf24420a2416b912035e0b7f5d7da/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:01.265171 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-0f4bcca8-06d1-4cb5-b9e1-208636f2ac49 None None] MFA Rules not processed for user `d5c945c9e9874c14b53c212c1d836e11`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:01.266478 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0f4bcca8-06d1-4cb5-b9e1-208636f2ac49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:01.267758 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f37cc0-3e09-47df-8b43-f83c96c32fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00338469c4d942389e2c5767bbdaee18, user_id=0ffdf24420a2416b912035e0b7f5d7da, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:01.270558 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f37cc0-3e09-47df-8b43-f83c96c32fb1 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:01.334329 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 998/1974] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:01.335215 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bcca8-06d1-4cb5-b9e1-208636f2ac49 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:01.347531 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bcca8-06d1-4cb5-b9e1-208636f2ac49 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:01.352188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f4d23-f415-4ac0-a75b-09d58a9b914d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:01.353115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f4d23-f415-4ac0-a75b-09d58a9b914d admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:01.353462 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f4d23-f415-4ac0-a75b-09d58a9b914d admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:01.353753 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f4d23-f415-4ac0-a75b-09d58a9b914d admin admin] PATH_INFO: `/v3/projects/0050d55d0fad4d90a10e12132f7f69ee/users/3b913d89578c4ccb8a2257057b88ca78/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:01.356023 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f4d23-f415-4ac0-a75b-09d58a9b914d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0050d55d0fad4d90a10e12132f7f69ee, user_id=3b913d89578c4ccb8a2257057b88ca78, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:01.358742 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f4d23-f415-4ac0-a75b-09d58a9b914d admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:01.376927 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 977/1975] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:01 2022] PUT /identity/v3/projects/00338469c4d942389e2c5767bbdaee18/users/0ffdf24420a2416b912035e0b7f5d7da/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:01.378036 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f37cc0-3e09-47df-8b43-f83c96c32fb1 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:01.386081 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 999/1976] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:01 2022] PUT /identity/v3/projects/0050d55d0fad4d90a10e12132f7f69ee/users/3b913d89578c4ccb8a2257057b88ca78/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:01.386081 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f4d23-f415-4ac0-a75b-09d58a9b914d admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:01.386081 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-7a6f4d23-f415-4ac0-a75b-09d58a9b914d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:01.470074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:01.477400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f37cc0-3e09-47df-8b43-f83c96c32fb1 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:01.481415 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f4d23-f415-4ac0-a75b-09d58a9b914d admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:01.486135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7aa1ab-f9b2-406b-9585-38922f7de62c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6c13596c2df438294bdac7c52ae8cb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:01.487068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7aa1ab-f9b2-406b-9585-38922f7de62c service glance] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:01.487362 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7aa1ab-f9b2-406b-9585-38922f7de62c service glance] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:01.487642 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7aa1ab-f9b2-406b-9585-38922f7de62c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:01.512054 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:01.558730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9cb99a-e089-4fb9-997c-e7fa86a97d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:01.559522 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9cb99a-e089-4fb9-997c-e7fa86a97d77 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:01.559879 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9cb99a-e089-4fb9-997c-e7fa86a97d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:01.560222 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9cb99a-e089-4fb9-997c-e7fa86a97d77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:01.561036 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9cb99a-e089-4fb9-997c-e7fa86a97d77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:33:01.561412 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9cb99a-e089-4fb9-997c-e7fa86a97d77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:01.563137 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9cb99a-e089-4fb9-997c-e7fa86a97d77 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:01.571004 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7aa1ab-f9b2-406b-9585-38922f7de62c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:01.571004 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7aa1ab-f9b2-406b-9585-38922f7de62c service glance] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:01.573634 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:01.577813 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 978/1977] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:33:01.579468 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab8377-deaf-4c07-85a8-e62dd452808a None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:01.579817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab8377-deaf-4c07-85a8-e62dd452808a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:01.580154 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab8377-deaf-4c07-85a8-e62dd452808a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:01.581988 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1000/1978] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:01.585849 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9439870-0244-493c-a412-cd97a130b629 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:01.586380 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9439870-0244-493c-a412-cd97a130b629 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:01.586864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9439870-0244-493c-a412-cd97a130b629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:01.613220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-f8ab8377-deaf-4c07-85a8-e62dd452808a None None] MFA Rules not processed for user `3b913d89578c4ccb8a2257057b88ca78`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:01.614560 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f8ab8377-deaf-4c07-85a8-e62dd452808a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:01.622138 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-d9439870-0244-493c-a412-cd97a130b629 None None] MFA Rules not processed for user `0ffdf24420a2416b912035e0b7f5d7da`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:01.623397 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d9439870-0244-493c-a412-cd97a130b629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:01.704118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 979/1979] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:01 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:01.712410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab8377-deaf-4c07-85a8-e62dd452808a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:01.712505 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1001/1980] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:01.713290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9439870-0244-493c-a412-cd97a130b629 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:01.722710 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab8377-deaf-4c07-85a8-e62dd452808a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:01.732510 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9439870-0244-493c-a412-cd97a130b629 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:01.734470 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8021810-083b-4748-bacb-580746e38b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:01.735433 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8021810-083b-4748-bacb-580746e38b7e admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:01.735603 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8021810-083b-4748-bacb-580746e38b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:01.735789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8021810-083b-4748-bacb-580746e38b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:01.736512 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8021810-083b-4748-bacb-580746e38b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:01.738460 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8021810-083b-4748-bacb-580746e38b7e admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:01.740062 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0fc92-b21e-4ea0-8fa7-04ebc4aeaf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:01.741140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0fc92-b21e-4ea0-8fa7-04ebc4aeaf27 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:01.741613 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0fc92-b21e-4ea0-8fa7-04ebc4aeaf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:01.742080 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0fc92-b21e-4ea0-8fa7-04ebc4aeaf27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:01.744844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0fc92-b21e-4ea0-8fa7-04ebc4aeaf27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:01.748488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0fc92-b21e-4ea0-8fa7-04ebc4aeaf27 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:01.857436 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1002/1981] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:33:01 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:01.859071 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0fc92-b21e-4ea0-8fa7-04ebc4aeaf27 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:01.861692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-3da0fc92-b21e-4ea0-8fa7-04ebc4aeaf27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:01.896341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 980/1982] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:33:01 2022] POST /identity/v3/projects => generated 385 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:01.897287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8021810-083b-4748-bacb-580746e38b7e admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:01.948196 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0fc92-b21e-4ea0-8fa7-04ebc4aeaf27 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:01.983200 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8021810-083b-4748-bacb-580746e38b7e admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:01.995842 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29dd7a3-eb8a-46bc-ae4b-c4531b9d4221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:01.996866 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29dd7a3-eb8a-46bc-ae4b-c4531b9d4221 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:01.997435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29dd7a3-eb8a-46bc-ae4b-c4531b9d4221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:01.997721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29dd7a3-eb8a-46bc-ae4b-c4531b9d4221 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:02.006863 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29dd7a3-eb8a-46bc-ae4b-c4531b9d4221 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:02.009312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29dd7a3-eb8a-46bc-ae4b-c4531b9d4221 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:02.010392 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ff3a8-b8eb-494d-b7f2-d9693fa7b9a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:02.011217 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ff3a8-b8eb-494d-b7f2-d9693fa7b9a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:02.011500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ff3a8-b8eb-494d-b7f2-d9693fa7b9a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:02.011770 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ff3a8-b8eb-494d-b7f2-d9693fa7b9a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:02.086105 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ff3a8-b8eb-494d-b7f2-d9693fa7b9a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:02.088074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ff3a8-b8eb-494d-b7f2-d9693fa7b9a9 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:02.104619 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1003/1983] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:01 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:02.105786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ff3a8-b8eb-494d-b7f2-d9693fa7b9a9 service cinder] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:02.116750 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ff3a8-b8eb-494d-b7f2-d9693fa7b9a9 service cinder] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:02.122102 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316396a-cad4-4281-a013-9f57722b0d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:02.122835 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316396a-cad4-4281-a013-9f57722b0d9b admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:02.123122 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316396a-cad4-4281-a013-9f57722b0d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:02.123434 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316396a-cad4-4281-a013-9f57722b0d9b admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:02.124137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316396a-cad4-4281-a013-9f57722b0d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:02.126383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316396a-cad4-4281-a013-9f57722b0d9b admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:02.128044 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:02.134395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 981/1984] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:33:01 2022] POST /identity/v3/users => generated 299 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:02.138570 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29dd7a3-eb8a-46bc-ae4b-c4531b9d4221 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:02.148984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29dd7a3-eb8a-46bc-ae4b-c4531b9d4221 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:02.154731 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8da3b-47ed-4779-890e-a1ae7d25a9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:02.155502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8da3b-47ed-4779-890e-a1ae7d25a9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:02.155855 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8da3b-47ed-4779-890e-a1ae7d25a9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:02.156204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8da3b-47ed-4779-890e-a1ae7d25a9c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:02.157052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8da3b-47ed-4779-890e-a1ae7d25a9c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:02.158704 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8da3b-47ed-4779-890e-a1ae7d25a9c7 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:02.169954 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 982/1985] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:33:02.174693 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8da3b-47ed-4779-890e-a1ae7d25a9c7 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:02.184961 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8da3b-47ed-4779-890e-a1ae7d25a9c7 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:02.190053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c962f7-2ec8-4d4e-b1a5-da15202a291d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:02.190991 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c962f7-2ec8-4d4e-b1a5-da15202a291d admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:02.191282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c962f7-2ec8-4d4e-b1a5-da15202a291d admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:02.191565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c962f7-2ec8-4d4e-b1a5-da15202a291d admin admin] PATH_INFO: `/v3/projects/1fadb930edac4574a945be18089d6232/users/537a57befe0f41099f72a406b3b7d3c2/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:02.208245 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c962f7-2ec8-4d4e-b1a5-da15202a291d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fadb930edac4574a945be18089d6232, user_id=537a57befe0f41099f72a406b3b7d3c2, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:02.210986 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c962f7-2ec8-4d4e-b1a5-da15202a291d admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:02.244447 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:02.251102 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1004/1986] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:33:02 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:02.257208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316396a-cad4-4281-a013-9f57722b0d9b admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:02.268116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316396a-cad4-4281-a013-9f57722b0d9b admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:02.277616 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852911c1-b3e0-40b4-8338-9d9c30dcd5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:02.278928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852911c1-b3e0-40b4-8338-9d9c30dcd5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:02.279211 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852911c1-b3e0-40b4-8338-9d9c30dcd5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:02.279488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852911c1-b3e0-40b4-8338-9d9c30dcd5d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:02.280221 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852911c1-b3e0-40b4-8338-9d9c30dcd5d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:02.281816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852911c1-b3e0-40b4-8338-9d9c30dcd5d9 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:02.296659 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1005/1987] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:33:02.302033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852911c1-b3e0-40b4-8338-9d9c30dcd5d9 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:02.312361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852911c1-b3e0-40b4-8338-9d9c30dcd5d9 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:02.318065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 983/1988] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:02 2022] PUT /identity/v3/projects/1fadb930edac4574a945be18089d6232/users/537a57befe0f41099f72a406b3b7d3c2/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:33:02.327862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c962f7-2ec8-4d4e-b1a5-da15202a291d admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:02.405298 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec70c58-e282-4cad-9d0b-ea5646932d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:02.406026 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec70c58-e282-4cad-9d0b-ea5646932d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:02.406487 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec70c58-e282-4cad-9d0b-ea5646932d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:02.406774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec70c58-e282-4cad-9d0b-ea5646932d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:02.407540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec70c58-e282-4cad-9d0b-ea5646932d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:02.409164 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec70c58-e282-4cad-9d0b-ea5646932d37 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:02.414729 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c962f7-2ec8-4d4e-b1a5-da15202a291d admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:02.420762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1006/1989] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:33:02.421872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841528e5-8c90-4556-ba0b-a9aefa71ce5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:02.426743 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec70c58-e282-4cad-9d0b-ea5646932d37 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:02.426867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841528e5-8c90-4556-ba0b-a9aefa71ce5b admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:02.426867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841528e5-8c90-4556-ba0b-a9aefa71ce5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:02.426867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841528e5-8c90-4556-ba0b-a9aefa71ce5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:02.426867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841528e5-8c90-4556-ba0b-a9aefa71ce5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:02.426867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841528e5-8c90-4556-ba0b-a9aefa71ce5b admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:02.440606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 984/1990] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:33:02.440606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841528e5-8c90-4556-ba0b-a9aefa71ce5b admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:02.448892 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841528e5-8c90-4556-ba0b-a9aefa71ce5b admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:02.453775 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e86a6-a316-424c-a98e-7367d44349d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:02.454755 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e86a6-a316-424c-a98e-7367d44349d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:02.455042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e86a6-a316-424c-a98e-7367d44349d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:02.455318 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e86a6-a316-424c-a98e-7367d44349d9 admin admin] PATH_INFO: `/v3/projects/1935e27754e841679ac0cefd7bbdc9f9/users/349d396a4c84440696e866f50e4b5783/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:02.471307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e86a6-a316-424c-a98e-7367d44349d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1935e27754e841679ac0cefd7bbdc9f9, user_id=349d396a4c84440696e866f50e4b5783, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:02.473987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e86a6-a316-424c-a98e-7367d44349d9 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:02.479653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec70c58-e282-4cad-9d0b-ea5646932d37 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:02.483561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe52266-6e38-4ae9-af34-07c2b5f53b02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:02.484390 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe52266-6e38-4ae9-af34-07c2b5f53b02 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:02.484672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe52266-6e38-4ae9-af34-07c2b5f53b02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:02.484957 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe52266-6e38-4ae9-af34-07c2b5f53b02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:02.486888 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2fe52266-6e38-4ae9-af34-07c2b5f53b02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:02.547127 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe52266-6e38-4ae9-af34-07c2b5f53b02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:02.548733 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe52266-6e38-4ae9-af34-07c2b5f53b02 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:02.558083 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1007/1991] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:02 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:02.558948 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe52266-6e38-4ae9-af34-07c2b5f53b02 service cinder] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:02.581835 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe52266-6e38-4ae9-af34-07c2b5f53b02 service cinder] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:02.587719 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 985/1992] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:02 2022] PUT /identity/v3/projects/1935e27754e841679ac0cefd7bbdc9f9/users/349d396a4c84440696e866f50e4b5783/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:02.595180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ff1c1-d5e7-4fef-bedf-4129c1ed64f2 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:02.595472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ff1c1-d5e7-4fef-bedf-4129c1ed64f2 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:02.595753 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ff1c1-d5e7-4fef-bedf-4129c1ed64f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:02.623189 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-f66ff1c1-d5e7-4fef-bedf-4129c1ed64f2 None None] MFA Rules not processed for user `349d396a4c84440696e866f50e4b5783`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:02.624611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f66ff1c1-d5e7-4fef-bedf-4129c1ed64f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:02.643471 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5229f-e6cf-44ff-8d63-ee0c521aad98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:02.644404 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5229f-e6cf-44ff-8d63-ee0c521aad98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:02.644683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5229f-e6cf-44ff-8d63-ee0c521aad98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:02.645105 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5229f-e6cf-44ff-8d63-ee0c521aad98 admin admin] PATH_INFO: `/v3/projects/1fadb930edac4574a945be18089d6232/users/537a57befe0f41099f72a406b3b7d3c2/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:02.647119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5229f-e6cf-44ff-8d63-ee0c521aad98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fadb930edac4574a945be18089d6232, user_id=537a57befe0f41099f72a406b3b7d3c2, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:02.650494 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5229f-e6cf-44ff-8d63-ee0c521aad98 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:02.661791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1008/1993] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:02 2022] PUT /identity/v3/projects/1fadb930edac4574a945be18089d6232/users/537a57befe0f41099f72a406b3b7d3c2/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:02.686934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9da1d6-f975-402d-b911-48f8e465af35 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:02.686934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9da1d6-f975-402d-b911-48f8e465af35 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:02.686934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9da1d6-f975-402d-b911-48f8e465af35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:02.723420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 986/1994] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:33:02.723591 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-1a9da1d6-f975-402d-b911-48f8e465af35 None None] MFA Rules not processed for user `537a57befe0f41099f72a406b3b7d3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:02.723591 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1a9da1d6-f975-402d-b911-48f8e465af35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:02.738958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ff1c1-d5e7-4fef-bedf-4129c1ed64f2 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:02.741183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f66ff1c1-d5e7-4fef-bedf-4129c1ed64f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:02.799687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1009/1995] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:02.801908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ff1c1-d5e7-4fef-bedf-4129c1ed64f2 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:02.824555 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:02.862763 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb2fbd-3714-478a-a60c-fe60c5ae492f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:02.863417 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb2fbd-3714-478a-a60c-fe60c5ae492f service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:02.863730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb2fbd-3714-478a-a60c-fe60c5ae492f service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:02.864136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb2fbd-3714-478a-a60c-fe60c5ae492f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:02.924824 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb2fbd-3714-478a-a60c-fe60c5ae492f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:02.926314 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb2fbd-3714-478a-a60c-fe60c5ae492f service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:02.944221 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 987/1996] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:02.965373 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9da1d6-f975-402d-b911-48f8e465af35 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:02.967082 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb2fbd-3714-478a-a60c-fe60c5ae492f service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:02.976297 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb2fbd-3714-478a-a60c-fe60c5ae492f service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:02.980480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a42e1f-28c6-4486-8b77-f7895a71340e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:02.980889 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a42e1f-28c6-4486-8b77-f7895a71340e service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:02.981078 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a42e1f-28c6-4486-8b77-f7895a71340e service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:02.981357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a42e1f-28c6-4486-8b77-f7895a71340e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:02.991072 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a42e1f-28c6-4486-8b77-f7895a71340e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:02.992527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a42e1f-28c6-4486-8b77-f7895a71340e service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:03.012720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 988/1997] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:33:03.044620 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9da1d6-f975-402d-b911-48f8e465af35 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:03.052057 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac0ddc-4a07-4f82-a88d-e399003f287c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:03.052956 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac0ddc-4a07-4f82-a88d-e399003f287c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:03.053345 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac0ddc-4a07-4f82-a88d-e399003f287c admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:03.053659 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac0ddc-4a07-4f82-a88d-e399003f287c admin admin] PATH_INFO: `/v3/users/349d396a4c84440696e866f50e4b5783` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:03.054950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac0ddc-4a07-4f82-a88d-e399003f287c admin admin] RBAC: Authorizing `identity:delete_user(user_id=349d396a4c84440696e866f50e4b5783)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:03.057699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac0ddc-4a07-4f82-a88d-e399003f287c admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:03.111323 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:03.411816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-40ac0ddc-4a07-4f82-a88d-e399003f287c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '349d396a4c84440696e866f50e4b5783'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:03.548955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-40ac0ddc-4a07-4f82-a88d-e399003f287c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '349d396a4c84440696e866f50e4b5783'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:03.559223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-40ac0ddc-4a07-4f82-a88d-e399003f287c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '349d396a4c84440696e866f50e4b5783'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:03.560040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-40ac0ddc-4a07-4f82-a88d-e399003f287c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '349d396a4c84440696e866f50e4b5783'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:03.572978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-40ac0ddc-4a07-4f82-a88d-e399003f287c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '349d396a4c84440696e866f50e4b5783'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:03.580487 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1010/1998] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:33:02 2022] DELETE /identity/v3/users/349d396a4c84440696e866f50e4b5783 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:33:03.585163 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5208b0-c5b7-492b-a627-38ae1d223229 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:03.585520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5208b0-c5b7-492b-a627-38ae1d223229 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:03.585806 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5208b0-c5b7-492b-a627-38ae1d223229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:03.606468 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-8a5208b0-c5b7-492b-a627-38ae1d223229 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:03.607852 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8a5208b0-c5b7-492b-a627-38ae1d223229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:03.683116 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 989/1999] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:03.684800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac0ddc-4a07-4f82-a88d-e399003f287c admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:03.686352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-40ac0ddc-4a07-4f82-a88d-e399003f287c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:03.754212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac0ddc-4a07-4f82-a88d-e399003f287c admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:03.758033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7f0e6-8ae3-4b8d-ae2e-565f5a77fe34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:03.758776 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7f0e6-8ae3-4b8d-ae2e-565f5a77fe34 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:03.759018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7f0e6-8ae3-4b8d-ae2e-565f5a77fe34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:03.759299 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7f0e6-8ae3-4b8d-ae2e-565f5a77fe34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:03.769803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7f0e6-8ae3-4b8d-ae2e-565f5a77fe34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:03.771635 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7f0e6-8ae3-4b8d-ae2e-565f5a77fe34 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:03.783562 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1011/2000] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:04.063893 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:04.404081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5208b0-c5b7-492b-a627-38ae1d223229 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:04.407945 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8a5208b0-c5b7-492b-a627-38ae1d223229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:04.448500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5208b0-c5b7-492b-a627-38ae1d223229 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:04.502741 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8acf5a-1e6e-4449-8725-beb2e73f3741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:04.504377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8acf5a-1e6e-4449-8725-beb2e73f3741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:04.504788 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8acf5a-1e6e-4449-8725-beb2e73f3741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:04.505257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8acf5a-1e6e-4449-8725-beb2e73f3741 admin admin] PATH_INFO: `/v3/projects/1935e27754e841679ac0cefd7bbdc9f9` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:04.511180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8acf5a-1e6e-4449-8725-beb2e73f3741 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1935e27754e841679ac0cefd7bbdc9f9)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:04.514551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8acf5a-1e6e-4449-8725-beb2e73f3741 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:04.680190 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7f0e6-8ae3-4b8d-ae2e-565f5a77fe34 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:04.682871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-4fd7f0e6-8ae3-4b8d-ae2e-565f5a77fe34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:04.737625 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-dd8acf5a-1e6e-4449-8725-beb2e73f3741 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1935e27754e841679ac0cefd7bbdc9f9', 'request_id': 'req-dd8acf5a-1e6e-4449-8725-beb2e73f3741'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:04.761825 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7f0e6-8ae3-4b8d-ae2e-565f5a77fe34 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:04.772190 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:04.833112 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c5d8c-e2ad-47ed-80c0-043c64d604bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:04.836455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c5d8c-e2ad-47ed-80c0-043c64d604bd service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:04.836455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c5d8c-e2ad-47ed-80c0-043c64d604bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:04.836455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c5d8c-e2ad-47ed-80c0-043c64d604bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:04.836840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9f2c5d8c-e2ad-47ed-80c0-043c64d604bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:04.863052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-dd8acf5a-1e6e-4449-8725-beb2e73f3741 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1935e27754e841679ac0cefd7bbdc9f9', 'request_id': 'req-dd8acf5a-1e6e-4449-8725-beb2e73f3741'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:04.876315 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-dd8acf5a-1e6e-4449-8725-beb2e73f3741 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1935e27754e841679ac0cefd7bbdc9f9', 'request_id': 'req-dd8acf5a-1e6e-4449-8725-beb2e73f3741'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:04.895012 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-dd8acf5a-1e6e-4449-8725-beb2e73f3741 admin admin] The token cache is being invalidate because project 1935e27754e841679ac0cefd7bbdc9f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:33:04.895592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-dd8acf5a-1e6e-4449-8725-beb2e73f3741 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:04.896392 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-dd8acf5a-1e6e-4449-8725-beb2e73f3741 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:04.898190 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 990/2001] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:33:04 2022] DELETE /identity/v3/projects/1935e27754e841679ac0cefd7bbdc9f9 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:04.903190 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f46dd7-3f1d-4084-8ced-cf05c504c3b3 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:04.903480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f46dd7-3f1d-4084-8ced-cf05c504c3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:04.903769 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f46dd7-3f1d-4084-8ced-cf05c504c3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:04.917922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c5d8c-e2ad-47ed-80c0-043c64d604bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:04.919482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c5d8c-e2ad-47ed-80c0-043c64d604bd service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:04.927407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-e6f46dd7-3f1d-4084-8ced-cf05c504c3b3 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:04.928675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e6f46dd7-3f1d-4084-8ced-cf05c504c3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:04.931784 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1012/2002] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:33:05.006054 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 991/2003] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:05.014488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12b9ea-bd47-4955-b7fe-058dce7b7081 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:05.014778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12b9ea-bd47-4955-b7fe-058dce7b7081 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:05.015058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12b9ea-bd47-4955-b7fe-058dce7b7081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:05.034564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6b12b9ea-bd47-4955-b7fe-058dce7b7081 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:05.035764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6b12b9ea-bd47-4955-b7fe-058dce7b7081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:05.044688 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1013/2004] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:05.049160 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f46dd7-3f1d-4084-8ced-cf05c504c3b3 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:05.061945 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f46dd7-3f1d-4084-8ced-cf05c504c3b3 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:05.066954 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ef6c8-0e53-4fce-a98e-2f28d773b068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:05.068386 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ef6c8-0e53-4fce-a98e-2f28d773b068 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:05.068606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ef6c8-0e53-4fce-a98e-2f28d773b068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:05.069398 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ef6c8-0e53-4fce-a98e-2f28d773b068 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:05.070032 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ef6c8-0e53-4fce-a98e-2f28d773b068 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:33:05.070370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ef6c8-0e53-4fce-a98e-2f28d773b068 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:05.072150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ef6c8-0e53-4fce-a98e-2f28d773b068 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:05.086310 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 992/2005] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:33:05.092644 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12b9ea-bd47-4955-b7fe-058dce7b7081 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:05.102095 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12b9ea-bd47-4955-b7fe-058dce7b7081 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:05.107818 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff6181-cda3-42d2-9c71-d0d33a0a8695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:05.108482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff6181-cda3-42d2-9c71-d0d33a0a8695 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:05.108970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff6181-cda3-42d2-9c71-d0d33a0a8695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:05.109267 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff6181-cda3-42d2-9c71-d0d33a0a8695 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:05.109974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff6181-cda3-42d2-9c71-d0d33a0a8695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:05.111991 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff6181-cda3-42d2-9c71-d0d33a0a8695 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:05.278762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1014/2006] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:33:05 2022] POST /identity/v3/projects => generated 377 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:33:05.281561 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ef6c8-0e53-4fce-a98e-2f28d773b068 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:05.296609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ef6c8-0e53-4fce-a98e-2f28d773b068 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:05.378098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da772d-6f2a-4b8f-9e68-df364b1141aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:05.378900 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da772d-6f2a-4b8f-9e68-df364b1141aa admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:05.379185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da772d-6f2a-4b8f-9e68-df364b1141aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:05.379472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da772d-6f2a-4b8f-9e68-df364b1141aa admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:05.380303 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da772d-6f2a-4b8f-9e68-df364b1141aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:05.382491 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da772d-6f2a-4b8f-9e68-df364b1141aa admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:05.533324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 993/2007] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:33:05 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:05.538316 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff6181-cda3-42d2-9c71-d0d33a0a8695 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:05.553856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff6181-cda3-42d2-9c71-d0d33a0a8695 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:05.559006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389983c-5196-4011-91e2-c22d83b4529c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:05.559695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389983c-5196-4011-91e2-c22d83b4529c admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:05.560130 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389983c-5196-4011-91e2-c22d83b4529c admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:05.560431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389983c-5196-4011-91e2-c22d83b4529c admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:05.561146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389983c-5196-4011-91e2-c22d83b4529c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:05.563004 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389983c-5196-4011-91e2-c22d83b4529c admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:05.577471 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1015/2008] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:05.578804 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da772d-6f2a-4b8f-9e68-df364b1141aa admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:05.594144 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da772d-6f2a-4b8f-9e68-df364b1141aa admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:05.601841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9cfd8-eabf-45e0-bc42-76f5ee02665e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:05.603070 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9cfd8-eabf-45e0-bc42-76f5ee02665e admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:05.603556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9cfd8-eabf-45e0-bc42-76f5ee02665e admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:05.604027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9cfd8-eabf-45e0-bc42-76f5ee02665e admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:05.605424 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9cfd8-eabf-45e0-bc42-76f5ee02665e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:05.608396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9cfd8-eabf-45e0-bc42-76f5ee02665e admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:05.627715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 994/2009] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:05.633075 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389983c-5196-4011-91e2-c22d83b4529c admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:05.651275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389983c-5196-4011-91e2-c22d83b4529c admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:05.658905 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf608a2-5a65-444b-b284-3596d3c7604b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:05.660817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf608a2-5a65-444b-b284-3596d3c7604b admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:05.661319 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf608a2-5a65-444b-b284-3596d3c7604b admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:05.662002 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf608a2-5a65-444b-b284-3596d3c7604b admin admin] PATH_INFO: `/v3/projects/bd3364f2b2fc49788daa0f4b03143015/users/e90624c2891e428cb237b08aa283042e/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:05.688644 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf608a2-5a65-444b-b284-3596d3c7604b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd3364f2b2fc49788daa0f4b03143015, user_id=e90624c2891e428cb237b08aa283042e, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:05.691437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf608a2-5a65-444b-b284-3596d3c7604b admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:05.812850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1016/2010] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:05 2022] PUT /identity/v3/projects/bd3364f2b2fc49788daa0f4b03143015/users/e90624c2891e428cb237b08aa283042e/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:05.827237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d7b3f-2b2a-4011-9c1c-872cf16eead5 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:05.827525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d7b3f-2b2a-4011-9c1c-872cf16eead5 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:05.827803 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d7b3f-2b2a-4011-9c1c-872cf16eead5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:05.857210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-8b2d7b3f-2b2a-4011-9c1c-872cf16eead5 None None] MFA Rules not processed for user `e90624c2891e428cb237b08aa283042e`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:05.858533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8b2d7b3f-2b2a-4011-9c1c-872cf16eead5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:05.931924 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 995/2011] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:05.946030 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf608a2-5a65-444b-b284-3596d3c7604b admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:05.946030 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9cf608a2-5a65-444b-b284-3596d3c7604b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:05.981125 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:06.010792 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf608a2-5a65-444b-b284-3596d3c7604b admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:06.015229 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb6ef7-3ca8-4aa0-8671-14e843c77552 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6c13596c2df438294bdac7c52ae8cb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:06.016028 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb6ef7-3ca8-4aa0-8671-14e843c77552 service glance] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:06.016373 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb6ef7-3ca8-4aa0-8671-14e843c77552 service glance] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:06.016748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb6ef7-3ca8-4aa0-8671-14e843c77552 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:06.027651 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb6ef7-3ca8-4aa0-8671-14e843c77552 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:06.030815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb6ef7-3ca8-4aa0-8671-14e843c77552 service glance] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:06.040561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1017/2012] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:07.880414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:08.717063 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:08.947104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:09.882653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:10.009398 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:10.400708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:10.471176 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:10.646296 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:10.688958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:10.814239 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:10.928622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:10.954493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:11.068791 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:11.105495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:11.283097 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:11.351777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:11.573827 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:11.620844 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:11.728870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:11.922967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:12.040020 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:12.836254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d7b3f-2b2a-4011-9c1c-872cf16eead5 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:12.838407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8b2d7b3f-2b2a-4011-9c1c-872cf16eead5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:12.964839 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d7b3f-2b2a-4011-9c1c-872cf16eead5 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:12.974882 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa0b2d-8bf6-432d-852e-403ef65651dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:12.975703 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa0b2d-8bf6-432d-852e-403ef65651dc service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:12.976052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa0b2d-8bf6-432d-852e-403ef65651dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:12.976415 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa0b2d-8bf6-432d-852e-403ef65651dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:12.979249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0faa0b2d-8bf6-432d-852e-403ef65651dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:13.136037 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa0b2d-8bf6-432d-852e-403ef65651dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:13.136037 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa0b2d-8bf6-432d-852e-403ef65651dc service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:13.142374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 996/2013] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:16.152046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb6ef7-3ca8-4aa0-8671-14e843c77552 service glance] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:16.157893 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6dcb6ef7-3ca8-4aa0-8671-14e843c77552 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:16.245742 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb6ef7-3ca8-4aa0-8671-14e843c77552 service glance] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:16.251015 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fe00f-738e-4262-8468-b08cbe0d5cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:16.251988 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fe00f-738e-4262-8468-b08cbe0d5cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:16.252275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fe00f-738e-4262-8468-b08cbe0d5cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:16.252616 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fe00f-738e-4262-8468-b08cbe0d5cc5 admin admin] PATH_INFO: `/v3/users/f001c05d64d04c69b28468b38fa98916` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:16.253894 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fe00f-738e-4262-8468-b08cbe0d5cc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f001c05d64d04c69b28468b38fa98916)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:16.255999 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fe00f-738e-4262-8468-b08cbe0d5cc5 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:16.637273 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-9c6fe00f-738e-4262-8468-b08cbe0d5cc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f001c05d64d04c69b28468b38fa98916'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:16.722190 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-9c6fe00f-738e-4262-8468-b08cbe0d5cc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f001c05d64d04c69b28468b38fa98916'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:16.729454 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:16.732254 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-9c6fe00f-738e-4262-8468-b08cbe0d5cc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f001c05d64d04c69b28468b38fa98916'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:16.733024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-9c6fe00f-738e-4262-8468-b08cbe0d5cc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f001c05d64d04c69b28468b38fa98916'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:16.756520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-9c6fe00f-738e-4262-8468-b08cbe0d5cc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f001c05d64d04c69b28468b38fa98916'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:16.764447 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1018/2014] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:33:16 2022] DELETE /identity/v3/users/f001c05d64d04c69b28468b38fa98916 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:16.769042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35e690-dad0-4a31-b752-e4f315f1860e None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:16.769515 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35e690-dad0-4a31-b752-e4f315f1860e None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:16.769671 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35e690-dad0-4a31-b752-e4f315f1860e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:16.794425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-3c35e690-dad0-4a31-b752-e4f315f1860e None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:16.794425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-3c35e690-dad0-4a31-b752-e4f315f1860e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:16.833032 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:16.893415 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 997/2015] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:16.903403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fe00f-738e-4262-8468-b08cbe0d5cc5 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:16.905081 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9c6fe00f-738e-4262-8468-b08cbe0d5cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:16.981715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fe00f-738e-4262-8468-b08cbe0d5cc5 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:16.985803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f32579-476e-4e63-ad0c-e3507796c8c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:16.986530 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f32579-476e-4e63-ad0c-e3507796c8c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:16.986998 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f32579-476e-4e63-ad0c-e3507796c8c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:16.987287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f32579-476e-4e63-ad0c-e3507796c8c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:16.998225 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f32579-476e-4e63-ad0c-e3507796c8c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:17.000807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f32579-476e-4e63-ad0c-e3507796c8c5 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:17.022383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1019/2016] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:18.282352 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35e690-dad0-4a31-b752-e4f315f1860e None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:18.284709 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-3c35e690-dad0-4a31-b752-e4f315f1860e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:18.300606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35e690-dad0-4a31-b752-e4f315f1860e None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:18.306703 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd1535-79d5-437f-91ff-a464eef41ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:18.307620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd1535-79d5-437f-91ff-a464eef41ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:18.307964 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd1535-79d5-437f-91ff-a464eef41ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:18.308572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd1535-79d5-437f-91ff-a464eef41ab2 admin admin] PATH_INFO: `/v3/users/ea27f45302e94a30894669344b028e13` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:18.309834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd1535-79d5-437f-91ff-a464eef41ab2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea27f45302e94a30894669344b028e13)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:18.310355 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbb087-3175-4601-a9ad-195265463fd6 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:18.310822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbb087-3175-4601-a9ad-195265463fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:18.311177 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbb087-3175-4601-a9ad-195265463fd6 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:18.312603 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd1535-79d5-437f-91ff-a464eef41ab2 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:18.313390 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1020/2017] 10.222.0.78 () {54 vars in 921 bytes} [Wed Jul 20 15:33:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:33:18.318896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbb087-3175-4601-a9ad-195265463fd6 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:18.320606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-a3cbb087-3175-4601-a9ad-195265463fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:18.341299 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbb087-3175-4601-a9ad-195265463fd6 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:18.356362 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280257c0-c0ee-4443-ae87-2f2e493870bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:18.357355 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280257c0-c0ee-4443-ae87-2f2e493870bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:18.357665 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280257c0-c0ee-4443-ae87-2f2e493870bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:18.357947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280257c0-c0ee-4443-ae87-2f2e493870bd admin admin] PATH_INFO: `/v3/projects/a85f65025aac4a20adcee358175d0aa1` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:18.359274 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280257c0-c0ee-4443-ae87-2f2e493870bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a85f65025aac4a20adcee358175d0aa1)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:18.362528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280257c0-c0ee-4443-ae87-2f2e493870bd admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:18.555211 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-280257c0-c0ee-4443-ae87-2f2e493870bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a85f65025aac4a20adcee358175d0aa1', 'request_id': 'req-280257c0-c0ee-4443-ae87-2f2e493870bd'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:18.556664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-9ecd1535-79d5-437f-91ff-a464eef41ab2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea27f45302e94a30894669344b028e13'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:18.653729 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-9ecd1535-79d5-437f-91ff-a464eef41ab2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea27f45302e94a30894669344b028e13'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:18.654551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-9ecd1535-79d5-437f-91ff-a464eef41ab2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea27f45302e94a30894669344b028e13'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:18.668973 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-9ecd1535-79d5-437f-91ff-a464eef41ab2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea27f45302e94a30894669344b028e13'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:18.676545 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-9ecd1535-79d5-437f-91ff-a464eef41ab2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea27f45302e94a30894669344b028e13'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:18.683302 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 998/2018] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:33:18 2022] DELETE /identity/v3/users/ea27f45302e94a30894669344b028e13 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:18.684508 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b8dd8-9261-4095-926d-f02e218ddfad None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:18.684794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b8dd8-9261-4095-926d-f02e218ddfad None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:18.685138 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b8dd8-9261-4095-926d-f02e218ddfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:18.704067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-410b8dd8-9261-4095-926d-f02e218ddfad None None] MFA Rules not processed for user `2192547babde4449a46f9cb37ed64524`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:18.705285 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-410b8dd8-9261-4095-926d-f02e218ddfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:18.743465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-280257c0-c0ee-4443-ae87-2f2e493870bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a85f65025aac4a20adcee358175d0aa1', 'request_id': 'req-280257c0-c0ee-4443-ae87-2f2e493870bd'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:18.751374 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-280257c0-c0ee-4443-ae87-2f2e493870bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a85f65025aac4a20adcee358175d0aa1', 'request_id': 'req-280257c0-c0ee-4443-ae87-2f2e493870bd'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:18.765115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-280257c0-c0ee-4443-ae87-2f2e493870bd admin admin] The token cache is being invalidate because project a85f65025aac4a20adcee358175d0aa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:33:18.765448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-280257c0-c0ee-4443-ae87-2f2e493870bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:18.766141 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-280257c0-c0ee-4443-ae87-2f2e493870bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:18.767813 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1021/2019] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:33:18 2022] DELETE /identity/v3/projects/a85f65025aac4a20adcee358175d0aa1 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:33:18.768604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280257c0-c0ee-4443-ae87-2f2e493870bd admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:18.770007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-280257c0-c0ee-4443-ae87-2f2e493870bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:18.793242 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 999/2020] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Jul 20 15:33:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:18.794762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9c511-1607-43a6-95f4-cae412023a8a None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:18.795047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9c511-1607-43a6-95f4-cae412023a8a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:18.795783 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9c511-1607-43a6-95f4-cae412023a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:18.821664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-85c9c511-1607-43a6-95f4-cae412023a8a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:18.822695 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-85c9c511-1607-43a6-95f4-cae412023a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:18.859972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280257c0-c0ee-4443-ae87-2f2e493870bd admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:18.863067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:18.868508 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:18.896743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1000/2021] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:18.898205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9c511-1607-43a6-95f4-cae412023a8a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:18.905567 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-85c9c511-1607-43a6-95f4-cae412023a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:18.917739 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9c511-1607-43a6-95f4-cae412023a8a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:18.922540 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b8f66-b841-4fc7-b990-6988530f70bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:18.923537 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b8f66-b841-4fc7-b990-6988530f70bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:18.923825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b8f66-b841-4fc7-b990-6988530f70bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:18.924107 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b8f66-b841-4fc7-b990-6988530f70bb admin admin] PATH_INFO: `/v3/users/3b913d89578c4ccb8a2257057b88ca78` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:18.924266 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40824fc7-4f29-41e4-90f7-1efe6363c0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:18.925155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40824fc7-4f29-41e4-90f7-1efe6363c0dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:18.925340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b8f66-b841-4fc7-b990-6988530f70bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b913d89578c4ccb8a2257057b88ca78)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:18.925445 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40824fc7-4f29-41e4-90f7-1efe6363c0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:18.925699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40824fc7-4f29-41e4-90f7-1efe6363c0dd admin admin] PATH_INFO: `/v3/users/48ce3bfba3724b07a657774662b8d6f3` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:18.926920 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40824fc7-4f29-41e4-90f7-1efe6363c0dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=48ce3bfba3724b07a657774662b8d6f3)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:18.927363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b8f66-b841-4fc7-b990-6988530f70bb admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:18.928829 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40824fc7-4f29-41e4-90f7-1efe6363c0dd admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:19.188752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-40824fc7-4f29-41e4-90f7-1efe6363c0dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48ce3bfba3724b07a657774662b8d6f3'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:19.189656 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-857b8f66-b841-4fc7-b990-6988530f70bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b913d89578c4ccb8a2257057b88ca78'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:19.319490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-857b8f66-b841-4fc7-b990-6988530f70bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b913d89578c4ccb8a2257057b88ca78'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:19.320197 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-857b8f66-b841-4fc7-b990-6988530f70bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b913d89578c4ccb8a2257057b88ca78'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:19.336269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-857b8f66-b841-4fc7-b990-6988530f70bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b913d89578c4ccb8a2257057b88ca78'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:19.345837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-857b8f66-b841-4fc7-b990-6988530f70bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b913d89578c4ccb8a2257057b88ca78'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:19.358387 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1001/2022] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:33:18 2022] DELETE /identity/v3/users/3b913d89578c4ccb8a2257057b88ca78 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:19.359247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b8f66-b841-4fc7-b990-6988530f70bb admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:19.360691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-857b8f66-b841-4fc7-b990-6988530f70bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:19.397362 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-40824fc7-4f29-41e4-90f7-1efe6363c0dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48ce3bfba3724b07a657774662b8d6f3'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:19.417410 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-40824fc7-4f29-41e4-90f7-1efe6363c0dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48ce3bfba3724b07a657774662b8d6f3'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:19.418446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-40824fc7-4f29-41e4-90f7-1efe6363c0dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48ce3bfba3724b07a657774662b8d6f3'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:19.433608 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-40824fc7-4f29-41e4-90f7-1efe6363c0dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48ce3bfba3724b07a657774662b8d6f3'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:19.440832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1022/2023] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:33:18 2022] DELETE /identity/v3/users/48ce3bfba3724b07a657774662b8d6f3 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:19.442254 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40824fc7-4f29-41e4-90f7-1efe6363c0dd admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:19.443966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-40824fc7-4f29-41e4-90f7-1efe6363c0dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:19.453363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b8f66-b841-4fc7-b990-6988530f70bb admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:19.538456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb486c-2b92-4d50-b09b-217dc87b025b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2192547babde4449a46f9cb37ed64524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:19.538456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb486c-2b92-4d50-b09b-217dc87b025b service placement] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:19.538456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb486c-2b92-4d50-b09b-217dc87b025b service placement] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:19.538456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb486c-2b92-4d50-b09b-217dc87b025b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:19.538908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-cedb486c-2b92-4d50-b09b-217dc87b025b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:19.538984 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40824fc7-4f29-41e4-90f7-1efe6363c0dd admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:19.541548 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfd872-0e02-42c5-9b2c-80e2b0629b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:19.542203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfd872-0e02-42c5-9b2c-80e2b0629b6a service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:19.542519 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfd872-0e02-42c5-9b2c-80e2b0629b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:19.542866 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfd872-0e02-42c5-9b2c-80e2b0629b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:19.545413 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e3dfd872-0e02-42c5-9b2c-80e2b0629b6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:19.561545 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb486c-2b92-4d50-b09b-217dc87b025b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:19.562406 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb486c-2b92-4d50-b09b-217dc87b025b service placement] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:19.573463 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1002/2024] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:19.630858 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dfd872-0e02-42c5-9b2c-80e2b0629b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:19.633062 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dfd872-0e02-42c5-9b2c-80e2b0629b6a service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:19.645432 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1023/2025] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:19.728380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:19.762987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:19.887599 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb486c-2b92-4d50-b09b-217dc87b025b service placement] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:19.889057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-cedb486c-2b92-4d50-b09b-217dc87b025b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:19.902994 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb486c-2b92-4d50-b09b-217dc87b025b service placement] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:19.908102 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:19.908976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:19.909290 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:19.909994 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1 admin admin] PATH_INFO: `/v3/projects/80caade751834762862c102c208a34e1` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:19.911917 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80caade751834762862c102c208a34e1)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:19.913188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:19.924770 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfd872-0e02-42c5-9b2c-80e2b0629b6a service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:19.926644 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e3dfd872-0e02-42c5-9b2c-80e2b0629b6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:19.951798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfd872-0e02-42c5-9b2c-80e2b0629b6a service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:19.966719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e4180-c736-4b49-83d1-06b034155c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:19.967642 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e4180-c736-4b49-83d1-06b034155c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:19.967970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e4180-c736-4b49-83d1-06b034155c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:19.968185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e4180-c736-4b49-83d1-06b034155c65 admin admin] PATH_INFO: `/v3/projects/0050d55d0fad4d90a10e12132f7f69ee` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:19.971698 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e4180-c736-4b49-83d1-06b034155c65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0050d55d0fad4d90a10e12132f7f69ee)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:19.973249 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e4180-c736-4b49-83d1-06b034155c65 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:20.190668 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80caade751834762862c102c208a34e1', 'request_id': 'req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:20.229658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-af9e4180-c736-4b49-83d1-06b034155c65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0050d55d0fad4d90a10e12132f7f69ee', 'request_id': 'req-af9e4180-c736-4b49-83d1-06b034155c65'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:20.336540 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80caade751834762862c102c208a34e1', 'request_id': 'req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:20.342392 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80caade751834762862c102c208a34e1', 'request_id': 'req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:20.354194 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1 admin admin] The token cache is being invalidate because project 80caade751834762862c102c208a34e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:33:20.354648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:20.357407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:20.357407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1003/2026] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:33:19 2022] DELETE /identity/v3/projects/80caade751834762862c102c208a34e1 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:20.358774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:20.364580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:20.436730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcaeb7e-7fb1-4e96-90ee-4c7c4b07bdc1 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:20.448085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-af9e4180-c736-4b49-83d1-06b034155c65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0050d55d0fad4d90a10e12132f7f69ee', 'request_id': 'req-af9e4180-c736-4b49-83d1-06b034155c65'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:20.460893 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-af9e4180-c736-4b49-83d1-06b034155c65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0050d55d0fad4d90a10e12132f7f69ee', 'request_id': 'req-af9e4180-c736-4b49-83d1-06b034155c65'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:20.481607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-af9e4180-c736-4b49-83d1-06b034155c65 admin admin] The token cache is being invalidate because project 0050d55d0fad4d90a10e12132f7f69ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:33:20.481949 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-af9e4180-c736-4b49-83d1-06b034155c65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:20.482607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-af9e4180-c736-4b49-83d1-06b034155c65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:20.484453 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1024/2027] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:33:19 2022] DELETE /identity/v3/projects/0050d55d0fad4d90a10e12132f7f69ee => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:20.485717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e27f2-50fc-4c9a-8e27-837557b72dde None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:20.486062 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e27f2-50fc-4c9a-8e27-837557b72dde None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:20.486404 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e27f2-50fc-4c9a-8e27-837557b72dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:20.505727 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330126f0-1737-4cdd-8bcf-67a6792d098e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:20.506638 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330126f0-1737-4cdd-8bcf-67a6792d098e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:20.506928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330126f0-1737-4cdd-8bcf-67a6792d098e admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:20.507209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330126f0-1737-4cdd-8bcf-67a6792d098e admin admin] PATH_INFO: `/v3/projects/47512a1c991a48cf96d492a4b25eec9f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:20.508964 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330126f0-1737-4cdd-8bcf-67a6792d098e admin admin] RBAC: Authorizing `identity:delete_project(project_id=47512a1c991a48cf96d492a4b25eec9f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:20.510992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330126f0-1737-4cdd-8bcf-67a6792d098e admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:20.517463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-205e27f2-50fc-4c9a-8e27-837557b72dde None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:20.519482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-205e27f2-50fc-4c9a-8e27-837557b72dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:20.544983 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1025/2028] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:20.548878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299a112-229b-4d5d-9942-24381ac2f581 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:20.548878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299a112-229b-4d5d-9942-24381ac2f581 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:20.548878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299a112-229b-4d5d-9942-24381ac2f581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:20.594868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-4299a112-229b-4d5d-9942-24381ac2f581 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:20.595901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-4299a112-229b-4d5d-9942-24381ac2f581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:20.616641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1026/2029] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:20.616641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38282922-0ac2-4e27-a6e9-c280bd5cc73e None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:20.616641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38282922-0ac2-4e27-a6e9-c280bd5cc73e None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:20.616641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38282922-0ac2-4e27-a6e9-c280bd5cc73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:20.632945 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-38282922-0ac2-4e27-a6e9-c280bd5cc73e None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:20.635198 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-38282922-0ac2-4e27-a6e9-c280bd5cc73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:20.657169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-330126f0-1737-4cdd-8bcf-67a6792d098e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47512a1c991a48cf96d492a4b25eec9f', 'request_id': 'req-330126f0-1737-4cdd-8bcf-67a6792d098e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:20.718652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1027/2030] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:20.718652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d73363-ba61-4ec2-a254-d6969d9ed801 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:20.718652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d73363-ba61-4ec2-a254-d6969d9ed801 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:20.718652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d73363-ba61-4ec2-a254-d6969d9ed801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:20.741066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-20d73363-ba61-4ec2-a254-d6969d9ed801 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:20.742529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-20d73363-ba61-4ec2-a254-d6969d9ed801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:20.748283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-330126f0-1737-4cdd-8bcf-67a6792d098e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47512a1c991a48cf96d492a4b25eec9f', 'request_id': 'req-330126f0-1737-4cdd-8bcf-67a6792d098e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:20.753970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1028/2031] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:20.758016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-330126f0-1737-4cdd-8bcf-67a6792d098e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47512a1c991a48cf96d492a4b25eec9f', 'request_id': 'req-330126f0-1737-4cdd-8bcf-67a6792d098e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:20.758294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d73363-ba61-4ec2-a254-d6969d9ed801 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:20.770568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d73363-ba61-4ec2-a254-d6969d9ed801 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:20.776325 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa85321-d5bf-4951-9f2d-8fc170d95b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:20.777065 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa85321-d5bf-4951-9f2d-8fc170d95b37 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:20.777405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa85321-d5bf-4951-9f2d-8fc170d95b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:20.777688 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa85321-d5bf-4951-9f2d-8fc170d95b37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:20.778483 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa85321-d5bf-4951-9f2d-8fc170d95b37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:33:20.786504 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa85321-d5bf-4951-9f2d-8fc170d95b37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:20.786700 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-330126f0-1737-4cdd-8bcf-67a6792d098e admin admin] The token cache is being invalidate because project 47512a1c991a48cf96d492a4b25eec9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:33:20.786700 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-330126f0-1737-4cdd-8bcf-67a6792d098e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:20.786955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa85321-d5bf-4951-9f2d-8fc170d95b37 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:20.787077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-330126f0-1737-4cdd-8bcf-67a6792d098e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:20.787077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1004/2032] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:33:20 2022] DELETE /identity/v3/projects/47512a1c991a48cf96d492a4b25eec9f => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:20.787077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330126f0-1737-4cdd-8bcf-67a6792d098e admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:20.787077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-330126f0-1737-4cdd-8bcf-67a6792d098e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:20.799994 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1029/2033] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:33:20.799994 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa85321-d5bf-4951-9f2d-8fc170d95b37 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:20.800274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330126f0-1737-4cdd-8bcf-67a6792d098e admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:20.800357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1aa85321-d5bf-4951-9f2d-8fc170d95b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:20.804787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2fc8b-403c-45ba-9d11-e3f331495164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:20.806057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2fc8b-403c-45ba-9d11-e3f331495164 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:20.806339 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2fc8b-403c-45ba-9d11-e3f331495164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:20.806619 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2fc8b-403c-45ba-9d11-e3f331495164 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:20.807529 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2fc8b-403c-45ba-9d11-e3f331495164 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:33:20.807840 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2fc8b-403c-45ba-9d11-e3f331495164 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:20.809768 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2fc8b-403c-45ba-9d11-e3f331495164 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:20.817926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa85321-d5bf-4951-9f2d-8fc170d95b37 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:20.823426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1005/2034] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:33:20.825145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2fc8b-403c-45ba-9d11-e3f331495164 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:20.825517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4dd2fc8b-403c-45ba-9d11-e3f331495164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:20.831270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a2c27-cd29-49d3-ac9b-1b2232e2609f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:20.832559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a2c27-cd29-49d3-ac9b-1b2232e2609f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:20.832961 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a2c27-cd29-49d3-ac9b-1b2232e2609f admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:20.833309 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a2c27-cd29-49d3-ac9b-1b2232e2609f admin admin] PATH_INFO: `/v3/users/0ffdf24420a2416b912035e0b7f5d7da` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:20.835014 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a2c27-cd29-49d3-ac9b-1b2232e2609f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ffdf24420a2416b912035e0b7f5d7da)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:20.837746 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a2c27-cd29-49d3-ac9b-1b2232e2609f admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:20.843775 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2fc8b-403c-45ba-9d11-e3f331495164 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:20.847120 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c3483-5415-44a6-b59c-2cbb572aa398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:20.848169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c3483-5415-44a6-b59c-2cbb572aa398 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:20.848652 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c3483-5415-44a6-b59c-2cbb572aa398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:20.849161 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c3483-5415-44a6-b59c-2cbb572aa398 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:20.850329 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c3483-5415-44a6-b59c-2cbb572aa398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:20.852917 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c3483-5415-44a6-b59c-2cbb572aa398 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:21.057391 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:21.065966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1006/2035] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:33:20 2022] POST /identity/v3/projects => generated 375 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:21.067027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c3483-5415-44a6-b59c-2cbb572aa398 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:21.138299 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f88a2c27-cd29-49d3-ac9b-1b2232e2609f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ffdf24420a2416b912035e0b7f5d7da'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:21.149948 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c3483-5415-44a6-b59c-2cbb572aa398 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:21.155051 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee0e88-f3ca-413f-9944-8c82cd2a0cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:21.155723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee0e88-f3ca-413f-9944-8c82cd2a0cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:21.156169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee0e88-f3ca-413f-9944-8c82cd2a0cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:21.156587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee0e88-f3ca-413f-9944-8c82cd2a0cf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:21.157376 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee0e88-f3ca-413f-9944-8c82cd2a0cf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:21.159288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee0e88-f3ca-413f-9944-8c82cd2a0cf0 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:21.276746 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f88a2c27-cd29-49d3-ac9b-1b2232e2609f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ffdf24420a2416b912035e0b7f5d7da'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:21.284326 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f88a2c27-cd29-49d3-ac9b-1b2232e2609f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ffdf24420a2416b912035e0b7f5d7da'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:21.285014 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f88a2c27-cd29-49d3-ac9b-1b2232e2609f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ffdf24420a2416b912035e0b7f5d7da'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:21.300132 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f88a2c27-cd29-49d3-ac9b-1b2232e2609f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ffdf24420a2416b912035e0b7f5d7da'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:21.305236 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:21.317259 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1007/2036] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:33:21 2022] POST /identity/v3/projects => generated 385 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:21.317259 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee0e88-f3ca-413f-9944-8c82cd2a0cf0 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:21.318342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1030/2037] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:33:20 2022] DELETE /identity/v3/users/0ffdf24420a2416b912035e0b7f5d7da => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:21.318342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a2c27-cd29-49d3-ac9b-1b2232e2609f admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:21.318515 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-97ee0e88-f3ca-413f-9944-8c82cd2a0cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:21.318600 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f88a2c27-cd29-49d3-ac9b-1b2232e2609f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:21.413450 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee0e88-f3ca-413f-9944-8c82cd2a0cf0 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:21.419660 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf2ced-ca7d-4af2-aaf5-d54b6f61359f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:21.420433 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf2ced-ca7d-4af2-aaf5-d54b6f61359f admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:21.420761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf2ced-ca7d-4af2-aaf5-d54b6f61359f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:21.421043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf2ced-ca7d-4af2-aaf5-d54b6f61359f admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:21.421828 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf2ced-ca7d-4af2-aaf5-d54b6f61359f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:21.422371 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a2c27-cd29-49d3-ac9b-1b2232e2609f admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:21.423702 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf2ced-ca7d-4af2-aaf5-d54b6f61359f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:21.431218 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75db89-5f72-4a58-93d3-3c9fef6dc91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:21.432654 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75db89-5f72-4a58-93d3-3c9fef6dc91c admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:21.433249 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75db89-5f72-4a58-93d3-3c9fef6dc91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:21.433817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75db89-5f72-4a58-93d3-3c9fef6dc91c admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:21.435221 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75db89-5f72-4a58-93d3-3c9fef6dc91c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:21.439161 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75db89-5f72-4a58-93d3-3c9fef6dc91c admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:21.523034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1008/2038] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:33:21 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:21.527916 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf2ced-ca7d-4af2-aaf5-d54b6f61359f admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:21.539240 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf2ced-ca7d-4af2-aaf5-d54b6f61359f admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:21.544642 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac60833-1c3a-4f6d-8768-ef4ec871236c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:21.545343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac60833-1c3a-4f6d-8768-ef4ec871236c admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:21.545626 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac60833-1c3a-4f6d-8768-ef4ec871236c admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:21.545903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac60833-1c3a-4f6d-8768-ef4ec871236c admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:21.546683 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac60833-1c3a-4f6d-8768-ef4ec871236c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:21.548383 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac60833-1c3a-4f6d-8768-ef4ec871236c admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:21.559568 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1009/2039] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:33:21.563185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac60833-1c3a-4f6d-8768-ef4ec871236c admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:21.569895 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1031/2040] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:33:21 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:21.573990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac60833-1c3a-4f6d-8768-ef4ec871236c admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:21.574493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75db89-5f72-4a58-93d3-3c9fef6dc91c admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:21.579326 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f71dc0-1bf9-47c5-be08-31b359d33822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:21.580289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f71dc0-1bf9-47c5-be08-31b359d33822 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:21.580289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f71dc0-1bf9-47c5-be08-31b359d33822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:21.580668 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f71dc0-1bf9-47c5-be08-31b359d33822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:21.581474 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f71dc0-1bf9-47c5-be08-31b359d33822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:21.583315 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f71dc0-1bf9-47c5-be08-31b359d33822 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:21.587791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75db89-5f72-4a58-93d3-3c9fef6dc91c admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:21.594054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de76478-58b4-4fe5-8cca-f40d89f07a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:21.595110 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de76478-58b4-4fe5-8cca-f40d89f07a8a admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:21.595396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de76478-58b4-4fe5-8cca-f40d89f07a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:21.595726 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de76478-58b4-4fe5-8cca-f40d89f07a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:21.596526 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de76478-58b4-4fe5-8cca-f40d89f07a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:21.598239 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de76478-58b4-4fe5-8cca-f40d89f07a8a admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:21.600226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1010/2041] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:33:21.604922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f71dc0-1bf9-47c5-be08-31b359d33822 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:21.616066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f71dc0-1bf9-47c5-be08-31b359d33822 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:21.622068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715d5d7-7dee-4f84-9dfc-a2fc74272a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:21.623033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715d5d7-7dee-4f84-9dfc-a2fc74272a9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:21.623463 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715d5d7-7dee-4f84-9dfc-a2fc74272a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:21.623750 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715d5d7-7dee-4f84-9dfc-a2fc74272a9f admin admin] PATH_INFO: `/v3/projects/94ddbdb125164a1fbbc85e5295c3ee93/users/192b8a31ff70484b9d0fc59c9c4ec075/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:21.625150 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1032/2042] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:21.628676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de76478-58b4-4fe5-8cca-f40d89f07a8a admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:21.643323 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de76478-58b4-4fe5-8cca-f40d89f07a8a admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:21.646494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715d5d7-7dee-4f84-9dfc-a2fc74272a9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=94ddbdb125164a1fbbc85e5295c3ee93, user_id=192b8a31ff70484b9d0fc59c9c4ec075, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:21.649823 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7b7a2-d6c8-4068-8cf1-6557cd9ff01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:21.650088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715d5d7-7dee-4f84-9dfc-a2fc74272a9f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:21.650582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7b7a2-d6c8-4068-8cf1-6557cd9ff01b admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:21.650870 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7b7a2-d6c8-4068-8cf1-6557cd9ff01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:21.651220 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7b7a2-d6c8-4068-8cf1-6557cd9ff01b admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:21.651881 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7b7a2-d6c8-4068-8cf1-6557cd9ff01b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:21.653582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7b7a2-d6c8-4068-8cf1-6557cd9ff01b admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:21.671819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1033/2043] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:21.681924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7b7a2-d6c8-4068-8cf1-6557cd9ff01b admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:21.694450 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7b7a2-d6c8-4068-8cf1-6557cd9ff01b admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:21.699525 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc302a6-82f3-4b31-a145-e001e0a2ca29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:21.700479 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc302a6-82f3-4b31-a145-e001e0a2ca29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:21.700766 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc302a6-82f3-4b31-a145-e001e0a2ca29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:21.701050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc302a6-82f3-4b31-a145-e001e0a2ca29 admin admin] PATH_INFO: `/v3/projects/29937f726169487981f22df2eadf07f4/users/ccc30a04d1de4114a0f42b90b1f17166/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:21.718488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc302a6-82f3-4b31-a145-e001e0a2ca29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29937f726169487981f22df2eadf07f4, user_id=ccc30a04d1de4114a0f42b90b1f17166, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:21.721639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc302a6-82f3-4b31-a145-e001e0a2ca29 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:21.761223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1011/2044] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:21 2022] PUT /identity/v3/projects/94ddbdb125164a1fbbc85e5295c3ee93/users/192b8a31ff70484b9d0fc59c9c4ec075/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:33:21.775368 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d614472-3ff4-45bd-80df-68230a49cd8d None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:21.776267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d614472-3ff4-45bd-80df-68230a49cd8d None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:21.776562 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d614472-3ff4-45bd-80df-68230a49cd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:21.810338 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-9d614472-3ff4-45bd-80df-68230a49cd8d None None] MFA Rules not processed for user `192b8a31ff70484b9d0fc59c9c4ec075`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:21.811698 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-9d614472-3ff4-45bd-80df-68230a49cd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:21.845739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1034/2045] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:21 2022] PUT /identity/v3/projects/29937f726169487981f22df2eadf07f4/users/ccc30a04d1de4114a0f42b90b1f17166/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:21.856386 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7decf-7d31-4ff7-b448-605d4f39dcc8 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:21.856669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7decf-7d31-4ff7-b448-605d4f39dcc8 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:21.856994 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7decf-7d31-4ff7-b448-605d4f39dcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:21.896882 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1012/2046] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:21.897961 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-d8d7decf-7d31-4ff7-b448-605d4f39dcc8 None None] MFA Rules not processed for user `ccc30a04d1de4114a0f42b90b1f17166`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:21.900429 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d8d7decf-7d31-4ff7-b448-605d4f39dcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:21.902171 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d614472-3ff4-45bd-80df-68230a49cd8d None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:21.991673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1035/2047] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:33:21.997365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7decf-7d31-4ff7-b448-605d4f39dcc8 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:22.003642 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d614472-3ff4-45bd-80df-68230a49cd8d None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:22.088290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7decf-7d31-4ff7-b448-605d4f39dcc8 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:22.088290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e8880-3095-4b94-a7f6-bad36d457641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:22.088835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaebc66-0ed6-4e16-9710-0d1ae222cd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:22.089598 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e8880-3095-4b94-a7f6-bad36d457641 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:22.089598 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e8880-3095-4b94-a7f6-bad36d457641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:22.089598 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e8880-3095-4b94-a7f6-bad36d457641 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:22.089876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaebc66-0ed6-4e16-9710-0d1ae222cd4e admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:22.089876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaebc66-0ed6-4e16-9710-0d1ae222cd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:22.089876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaebc66-0ed6-4e16-9710-0d1ae222cd4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:22.090094 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e8880-3095-4b94-a7f6-bad36d457641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:22.090175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaebc66-0ed6-4e16-9710-0d1ae222cd4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:22.090252 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e8880-3095-4b94-a7f6-bad36d457641 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:22.090366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaebc66-0ed6-4e16-9710-0d1ae222cd4e admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:22.284644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1013/2048] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:33:21 2022] POST /identity/v3/projects => generated 375 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:33:22.285657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaebc66-0ed6-4e16-9710-0d1ae222cd4e admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:22.287609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-aeaebc66-0ed6-4e16-9710-0d1ae222cd4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:22.356544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaebc66-0ed6-4e16-9710-0d1ae222cd4e admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:22.361657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8bed2-0e16-485d-9063-651295f22443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:22.362560 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8bed2-0e16-485d-9063-651295f22443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:22.371225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8bed2-0e16-485d-9063-651295f22443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:22.371225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8bed2-0e16-485d-9063-651295f22443 admin admin] PATH_INFO: `/v3/projects/00338469c4d942389e2c5767bbdaee18` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:22.371225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8bed2-0e16-485d-9063-651295f22443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00338469c4d942389e2c5767bbdaee18)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:22.371225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8bed2-0e16-485d-9063-651295f22443 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:22.371576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1036/2049] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:33:21 2022] POST /identity/v3/projects => generated 383 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:33:22.371576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e8880-3095-4b94-a7f6-bad36d457641 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:22.441737 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e8880-3095-4b94-a7f6-bad36d457641 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:22.450238 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed2ac7-9216-43a9-9814-6ac6af8e3474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:22.450238 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed2ac7-9216-43a9-9814-6ac6af8e3474 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:22.450238 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed2ac7-9216-43a9-9814-6ac6af8e3474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:22.450238 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed2ac7-9216-43a9-9814-6ac6af8e3474 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:22.450662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed2ac7-9216-43a9-9814-6ac6af8e3474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:22.450662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed2ac7-9216-43a9-9814-6ac6af8e3474 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:22.533446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-c7c8bed2-0e16-485d-9063-651295f22443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00338469c4d942389e2c5767bbdaee18', 'request_id': 'req-c7c8bed2-0e16-485d-9063-651295f22443'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:22.554496 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1037/2050] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:33:22 2022] POST /identity/v3/users => generated 294 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:22.555370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed2ac7-9216-43a9-9814-6ac6af8e3474 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:22.565343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed2ac7-9216-43a9-9814-6ac6af8e3474 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:22.619557 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-c7c8bed2-0e16-485d-9063-651295f22443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00338469c4d942389e2c5767bbdaee18', 'request_id': 'req-c7c8bed2-0e16-485d-9063-651295f22443'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:22.625290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b95b8c-b691-4747-adab-be8849937a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:22.625996 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b95b8c-b691-4747-adab-be8849937a8f admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:22.626306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b95b8c-b691-4747-adab-be8849937a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:22.626591 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b95b8c-b691-4747-adab-be8849937a8f admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:22.626719 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-c7c8bed2-0e16-485d-9063-651295f22443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00338469c4d942389e2c5767bbdaee18', 'request_id': 'req-c7c8bed2-0e16-485d-9063-651295f22443'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:22.627296 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b95b8c-b691-4747-adab-be8849937a8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:22.629305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b95b8c-b691-4747-adab-be8849937a8f admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:22.639584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-c7c8bed2-0e16-485d-9063-651295f22443 admin admin] The token cache is being invalidate because project 00338469c4d942389e2c5767bbdaee18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:33:22.639875 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-c7c8bed2-0e16-485d-9063-651295f22443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:22.640574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-c7c8bed2-0e16-485d-9063-651295f22443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:22.642137 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1014/2051] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:33:22 2022] DELETE /identity/v3/projects/00338469c4d942389e2c5767bbdaee18 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:22.642931 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8bed2-0e16-485d-9063-651295f22443 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:22.644215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-c7c8bed2-0e16-485d-9063-651295f22443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:22.657704 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8bed2-0e16-485d-9063-651295f22443 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:22.662760 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953ba42-bd49-4b99-8271-acfe20c332b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:22.663435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953ba42-bd49-4b99-8271-acfe20c332b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:22.663718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953ba42-bd49-4b99-8271-acfe20c332b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:22.663990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953ba42-bd49-4b99-8271-acfe20c332b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:22.664764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953ba42-bd49-4b99-8271-acfe20c332b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:22.666596 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953ba42-bd49-4b99-8271-acfe20c332b5 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:22.676804 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1015/2052] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:22.677606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953ba42-bd49-4b99-8271-acfe20c332b5 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:22.679286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5953ba42-bd49-4b99-8271-acfe20c332b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:22.697028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953ba42-bd49-4b99-8271-acfe20c332b5 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:22.702436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af302a-6a27-43a0-ac49-cb52ff2462ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:22.703850 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af302a-6a27-43a0-ac49-cb52ff2462ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:22.704714 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af302a-6a27-43a0-ac49-cb52ff2462ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:22.704714 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af302a-6a27-43a0-ac49-cb52ff2462ce admin admin] PATH_INFO: `/v3/users/537a57befe0f41099f72a406b3b7d3c2` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:22.708789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af302a-6a27-43a0-ac49-cb52ff2462ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=537a57befe0f41099f72a406b3b7d3c2)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:22.708789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af302a-6a27-43a0-ac49-cb52ff2462ce admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:22.745501 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1038/2053] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:33:22 2022] POST /identity/v3/users => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:22.746407 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b95b8c-b691-4747-adab-be8849937a8f admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:22.756560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b95b8c-b691-4747-adab-be8849937a8f admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:22.762740 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f98a2-3e4d-4350-b7b1-de9ceff285af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:22.763684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f98a2-3e4d-4350-b7b1-de9ceff285af admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:22.763970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f98a2-3e4d-4350-b7b1-de9ceff285af admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:22.764248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f98a2-3e4d-4350-b7b1-de9ceff285af admin admin] PATH_INFO: `/v3/projects/116f245900cf4d22b211ae8c22e23882/users/49b85d860db44e2aad1173dde389bfe8/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:22.781147 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f98a2-3e4d-4350-b7b1-de9ceff285af admin admin] RBAC: Authorizing `identity:create_grant(project_id=116f245900cf4d22b211ae8c22e23882, user_id=49b85d860db44e2aad1173dde389bfe8, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:22.784323 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f98a2-3e4d-4350-b7b1-de9ceff285af admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:22.868705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1039/2054] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:22 2022] PUT /identity/v3/projects/116f245900cf4d22b211ae8c22e23882/users/49b85d860db44e2aad1173dde389bfe8/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:22.870813 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f98a2-3e4d-4350-b7b1-de9ceff285af admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:22.871234 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8f5f98a2-3e4d-4350-b7b1-de9ceff285af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:22.949261 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f98a2-3e4d-4350-b7b1-de9ceff285af admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:22.953347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d5af302a-6a27-43a0-ac49-cb52ff2462ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '537a57befe0f41099f72a406b3b7d3c2'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:22.961919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c331ce6-56f0-4c01-a5c8-1d9c2a996fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:22.962695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c331ce6-56f0-4c01-a5c8-1d9c2a996fdb admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:22.963020 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c331ce6-56f0-4c01-a5c8-1d9c2a996fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:22.963330 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c331ce6-56f0-4c01-a5c8-1d9c2a996fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:22.964374 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c331ce6-56f0-4c01-a5c8-1d9c2a996fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:22.966739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c331ce6-56f0-4c01-a5c8-1d9c2a996fdb admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:22.967295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:22.977533 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1040/2055] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:22.978450 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c331ce6-56f0-4c01-a5c8-1d9c2a996fdb admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:22.994584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c331ce6-56f0-4c01-a5c8-1d9c2a996fdb admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:23.001465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaea891-9b60-4d74-9060-4c837e52e8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:23.002156 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaea891-9b60-4d74-9060-4c837e52e8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:23.002541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaea891-9b60-4d74-9060-4c837e52e8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:23.002827 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaea891-9b60-4d74-9060-4c837e52e8a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:23.003607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaea891-9b60-4d74-9060-4c837e52e8a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:23.005928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaea891-9b60-4d74-9060-4c837e52e8a5 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:23.017099 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1041/2056] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:23.017970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaea891-9b60-4d74-9060-4c837e52e8a5 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:23.025492 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d5af302a-6a27-43a0-ac49-cb52ff2462ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '537a57befe0f41099f72a406b3b7d3c2'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:23.026371 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d5af302a-6a27-43a0-ac49-cb52ff2462ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '537a57befe0f41099f72a406b3b7d3c2'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:23.028825 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaea891-9b60-4d74-9060-4c837e52e8a5 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:23.036529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66715c19-8cb8-4d94-9409-72714931ffe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:23.037947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66715c19-8cb8-4d94-9409-72714931ffe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:23.038484 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66715c19-8cb8-4d94-9409-72714931ffe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:23.038999 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66715c19-8cb8-4d94-9409-72714931ffe0 admin admin] PATH_INFO: `/v3/projects/b3e3804d06ad496da11844d35def6204/users/ad50c121d1f04c98a680409bdd9b9336/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:23.048905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d5af302a-6a27-43a0-ac49-cb52ff2462ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '537a57befe0f41099f72a406b3b7d3c2'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:23.058586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d5af302a-6a27-43a0-ac49-cb52ff2462ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '537a57befe0f41099f72a406b3b7d3c2'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:23.066547 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66715c19-8cb8-4d94-9409-72714931ffe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3e3804d06ad496da11844d35def6204, user_id=ad50c121d1f04c98a680409bdd9b9336, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:23.067317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1016/2057] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:33:22 2022] DELETE /identity/v3/users/537a57befe0f41099f72a406b3b7d3c2 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:23.068109 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af302a-6a27-43a0-ac49-cb52ff2462ce admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:23.069182 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66715c19-8cb8-4d94-9409-72714931ffe0 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:23.069759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d5af302a-6a27-43a0-ac49-cb52ff2462ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:23.123094 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:23.160383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1042/2058] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:23 2022] PUT /identity/v3/projects/b3e3804d06ad496da11844d35def6204/users/ad50c121d1f04c98a680409bdd9b9336/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:23.161307 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66715c19-8cb8-4d94-9409-72714931ffe0 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:23.163168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-66715c19-8cb8-4d94-9409-72714931ffe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:23.183086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af302a-6a27-43a0-ac49-cb52ff2462ce admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:23.279199 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f8670-c7cf-48ae-8877-f0c46aec7727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:23.279199 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f8670-c7cf-48ae-8877-f0c46aec7727 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:23.279520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66715c19-8cb8-4d94-9409-72714931ffe0 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:23.279610 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f8670-c7cf-48ae-8877-f0c46aec7727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:23.279610 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f8670-c7cf-48ae-8877-f0c46aec7727 admin admin] PATH_INFO: `/v3/projects/116f245900cf4d22b211ae8c22e23882/users/49b85d860db44e2aad1173dde389bfe8/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:23.279610 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f8670-c7cf-48ae-8877-f0c46aec7727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=116f245900cf4d22b211ae8c22e23882, user_id=49b85d860db44e2aad1173dde389bfe8, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:23.279610 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f8670-c7cf-48ae-8877-f0c46aec7727 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:23.279925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c7e9b-9b28-41be-b04a-e0b98e6141cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:23.280024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c7e9b-9b28-41be-b04a-e0b98e6141cf admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:23.280318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c7e9b-9b28-41be-b04a-e0b98e6141cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:23.280645 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c7e9b-9b28-41be-b04a-e0b98e6141cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:23.281356 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c7e9b-9b28-41be-b04a-e0b98e6141cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:23.283312 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c7e9b-9b28-41be-b04a-e0b98e6141cf admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:23.303088 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1043/2059] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:23.303318 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1017/2060] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:23 2022] PUT /identity/v3/projects/116f245900cf4d22b211ae8c22e23882/users/49b85d860db44e2aad1173dde389bfe8/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:23.310554 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c7e9b-9b28-41be-b04a-e0b98e6141cf admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:23.315093 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f850b7-2f5c-445d-90ed-b05a75d8c33c None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:23.315486 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f850b7-2f5c-445d-90ed-b05a75d8c33c None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:23.315852 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f850b7-2f5c-445d-90ed-b05a75d8c33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:23.325111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c7e9b-9b28-41be-b04a-e0b98e6141cf admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:23.358921 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-e5f850b7-2f5c-445d-90ed-b05a75d8c33c None None] MFA Rules not processed for user `49b85d860db44e2aad1173dde389bfe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:23.360213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e5f850b7-2f5c-445d-90ed-b05a75d8c33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:23.422286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432023d-82a9-4d4c-9f1a-c59619fd3ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:23.423590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432023d-82a9-4d4c-9f1a-c59619fd3ed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:23.424602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432023d-82a9-4d4c-9f1a-c59619fd3ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:23.424602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432023d-82a9-4d4c-9f1a-c59619fd3ed6 admin admin] PATH_INFO: `/v3/projects/b3e3804d06ad496da11844d35def6204/users/ad50c121d1f04c98a680409bdd9b9336/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:23.427736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432023d-82a9-4d4c-9f1a-c59619fd3ed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3e3804d06ad496da11844d35def6204, user_id=ad50c121d1f04c98a680409bdd9b9336, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:23.431987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432023d-82a9-4d4c-9f1a-c59619fd3ed6 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:23.452779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1044/2061] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:23 2022] PUT /identity/v3/projects/b3e3804d06ad496da11844d35def6204/users/ad50c121d1f04c98a680409bdd9b9336/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:23.464460 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1018/2062] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:33:23.474865 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05999ec6-6adb-4c4c-9114-8e67cf9943de None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:23.475396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05999ec6-6adb-4c4c-9114-8e67cf9943de None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:23.475879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05999ec6-6adb-4c4c-9114-8e67cf9943de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:23.488692 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f850b7-2f5c-445d-90ed-b05a75d8c33c None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:23.493722 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e5f850b7-2f5c-445d-90ed-b05a75d8c33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:23.517633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-05999ec6-6adb-4c4c-9114-8e67cf9943de None None] MFA Rules not processed for user `ad50c121d1f04c98a680409bdd9b9336`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:23.519872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-05999ec6-6adb-4c4c-9114-8e67cf9943de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:23.569950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f850b7-2f5c-445d-90ed-b05a75d8c33c None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:23.630065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8c2bd-2c62-47f4-9064-50bda32a0108 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:23.631097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8c2bd-2c62-47f4-9064-50bda32a0108 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:23.631384 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8c2bd-2c62-47f4-9064-50bda32a0108 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:23.631679 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8c2bd-2c62-47f4-9064-50bda32a0108 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:23.632254 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1045/2063] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:23.637202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-21d8c2bd-2c62-47f4-9064-50bda32a0108 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:23.637345 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05999ec6-6adb-4c4c-9114-8e67cf9943de None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:23.640626 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-05999ec6-6adb-4c4c-9114-8e67cf9943de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:23.650668 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:23.705715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8c2bd-2c62-47f4-9064-50bda32a0108 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:23.711776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8c2bd-2c62-47f4-9064-50bda32a0108 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:23.715846 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05999ec6-6adb-4c4c-9114-8e67cf9943de None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:23.720912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e1410-91ec-430c-b27d-ce8aeceb8c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:23.721524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1019/2064] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:33:23.721794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e1410-91ec-430c-b27d-ce8aeceb8c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:23.722142 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e1410-91ec-430c-b27d-ce8aeceb8c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:23.722431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8c2bd-2c62-47f4-9064-50bda32a0108 service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:23.722604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e1410-91ec-430c-b27d-ce8aeceb8c85 admin admin] PATH_INFO: `/v3/projects/1fadb930edac4574a945be18089d6232` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:23.723767 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-21d8c2bd-2c62-47f4-9064-50bda32a0108 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:23.723932 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e1410-91ec-430c-b27d-ce8aeceb8c85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fadb930edac4574a945be18089d6232)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:23.726056 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e1410-91ec-430c-b27d-ce8aeceb8c85 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:23.739775 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8c2bd-2c62-47f4-9064-50bda32a0108 service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:23.744557 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:23.748594 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3a31d-82e0-4971-9c00-de74839aad37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:23.748594 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3a31d-82e0-4971-9c00-de74839aad37 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:23.748594 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3a31d-82e0-4971-9c00-de74839aad37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:23.748594 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3a31d-82e0-4971-9c00-de74839aad37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:23.749169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-cbb3a31d-82e0-4971-9c00-de74839aad37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:23.819167 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb3a31d-82e0-4971-9c00-de74839aad37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:23.823126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb3a31d-82e0-4971-9c00-de74839aad37 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:23.840197 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1020/2065] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:23.863231 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-116e1410-91ec-430c-b27d-ce8aeceb8c85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fadb930edac4574a945be18089d6232', 'request_id': 'req-116e1410-91ec-430c-b27d-ce8aeceb8c85'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:23.994084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-116e1410-91ec-430c-b27d-ce8aeceb8c85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fadb930edac4574a945be18089d6232', 'request_id': 'req-116e1410-91ec-430c-b27d-ce8aeceb8c85'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:24.001988 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-116e1410-91ec-430c-b27d-ce8aeceb8c85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fadb930edac4574a945be18089d6232', 'request_id': 'req-116e1410-91ec-430c-b27d-ce8aeceb8c85'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:24.014236 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-116e1410-91ec-430c-b27d-ce8aeceb8c85 admin admin] The token cache is being invalidate because project 1fadb930edac4574a945be18089d6232 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:33:24.014709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-116e1410-91ec-430c-b27d-ce8aeceb8c85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:24.015423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-116e1410-91ec-430c-b27d-ce8aeceb8c85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:24.017393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1046/2066] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:33:23 2022] DELETE /identity/v3/projects/1fadb930edac4574a945be18089d6232 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:24.024544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9e838-8dda-4b0b-9171-8967887faaa3 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:24.026391 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9e838-8dda-4b0b-9171-8967887faaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:24.026845 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9e838-8dda-4b0b-9171-8967887faaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:24.051933 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-64e9e838-8dda-4b0b-9171-8967887faaa3 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:24.053520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-64e9e838-8dda-4b0b-9171-8967887faaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:24.136088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1021/2067] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:24.147100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a9d02-0328-4907-b141-219dd14b8575 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:24.147588 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a9d02-0328-4907-b141-219dd14b8575 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:24.148012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a9d02-0328-4907-b141-219dd14b8575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:24.172083 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-bf3a9d02-0328-4907-b141-219dd14b8575 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:24.173370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-bf3a9d02-0328-4907-b141-219dd14b8575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:24.183068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1047/2068] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:24.189956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9e838-8dda-4b0b-9171-8967887faaa3 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:24.204771 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9e838-8dda-4b0b-9171-8967887faaa3 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:24.210253 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7fb65-47f2-4aab-ae86-57c27c0e95b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:24.210970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7fb65-47f2-4aab-ae86-57c27c0e95b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:24.211268 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7fb65-47f2-4aab-ae86-57c27c0e95b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:24.211576 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7fb65-47f2-4aab-ae86-57c27c0e95b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:24.212377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed7fb65-47f2-4aab-ae86-57c27c0e95b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:33:24.212687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed7fb65-47f2-4aab-ae86-57c27c0e95b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:24.214799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed7fb65-47f2-4aab-ae86-57c27c0e95b5 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:24.227626 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:24.235541 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1022/2069] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:33:24.241350 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a9d02-0328-4907-b141-219dd14b8575 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:24.267183 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a9d02-0328-4907-b141-219dd14b8575 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:24.276299 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90d056-028f-492f-a813-a061abdb80e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:24.277224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90d056-028f-492f-a813-a061abdb80e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:24.277957 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90d056-028f-492f-a813-a061abdb80e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:24.278506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90d056-028f-492f-a813-a061abdb80e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:24.279713 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e90d056-028f-492f-a813-a061abdb80e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:24.282999 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e90d056-028f-492f-a813-a061abdb80e5 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:24.340410 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:24.429018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1048/2070] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:33:24 2022] POST /identity/v3/projects => generated 393 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:24.439823 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7fb65-47f2-4aab-ae86-57c27c0e95b5 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:24.449972 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7fb65-47f2-4aab-ae86-57c27c0e95b5 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:24.510685 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d94d17-7a4d-490e-a393-862338184ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:24.511363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d94d17-7a4d-490e-a393-862338184ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:24.511816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d94d17-7a4d-490e-a393-862338184ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:24.512058 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d94d17-7a4d-490e-a393-862338184ff4 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:24.512774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d94d17-7a4d-490e-a393-862338184ff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:24.514695 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d94d17-7a4d-490e-a393-862338184ff4 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:24.615154 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1023/2071] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:33:24 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:24.620110 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90d056-028f-492f-a813-a061abdb80e5 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:24.631707 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90d056-028f-492f-a813-a061abdb80e5 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:24.637338 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b41142-72ee-4d14-b2cd-d466f76c5f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:24.638198 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b41142-72ee-4d14-b2cd-d466f76c5f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:24.638198 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b41142-72ee-4d14-b2cd-d466f76c5f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:24.638513 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b41142-72ee-4d14-b2cd-d466f76c5f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:24.639226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b41142-72ee-4d14-b2cd-d466f76c5f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:24.640832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b41142-72ee-4d14-b2cd-d466f76c5f58 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:24.651935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1049/2072] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:24.656475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d94d17-7a4d-490e-a393-862338184ff4 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:24.669911 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d94d17-7a4d-490e-a393-862338184ff4 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:24.676391 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8909dd9-39f6-49ed-8f4c-ac444e2a5603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:24.676391 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8909dd9-39f6-49ed-8f4c-ac444e2a5603 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:24.676671 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8909dd9-39f6-49ed-8f4c-ac444e2a5603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:24.676902 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8909dd9-39f6-49ed-8f4c-ac444e2a5603 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:24.677653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8909dd9-39f6-49ed-8f4c-ac444e2a5603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:24.679295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8909dd9-39f6-49ed-8f4c-ac444e2a5603 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:24.691642 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1024/2073] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:33:24.699377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b41142-72ee-4d14-b2cd-d466f76c5f58 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:24.710323 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b41142-72ee-4d14-b2cd-d466f76c5f58 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:24.715581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a4224-6236-4c00-a104-d96237ccb490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:24.716468 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a4224-6236-4c00-a104-d96237ccb490 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:24.716748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a4224-6236-4c00-a104-d96237ccb490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:24.717028 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a4224-6236-4c00-a104-d96237ccb490 admin admin] PATH_INFO: `/v3/projects/94739478923d405d920943480fb53e29/users/69818cb63a374ee8b277790294e70ef9/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:24.757410 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a4224-6236-4c00-a104-d96237ccb490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94739478923d405d920943480fb53e29, user_id=69818cb63a374ee8b277790294e70ef9, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:24.760941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a4224-6236-4c00-a104-d96237ccb490 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:24.885026 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1050/2074] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:24 2022] PUT /identity/v3/projects/94739478923d405d920943480fb53e29/users/69818cb63a374ee8b277790294e70ef9/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:33:24.904927 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3ebc4-77d4-41a0-80ee-aebae06191e8 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:24.905245 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3ebc4-77d4-41a0-80ee-aebae06191e8 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:24.905537 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3ebc4-77d4-41a0-80ee-aebae06191e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:24.941026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-0ee3ebc4-77d4-41a0-80ee-aebae06191e8 None None] MFA Rules not processed for user `69818cb63a374ee8b277790294e70ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:24.942966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0ee3ebc4-77d4-41a0-80ee-aebae06191e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:25.037778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:25.040721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1025/2075] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:25.048971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a4224-6236-4c00-a104-d96237ccb490 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:25.135375 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a4224-6236-4c00-a104-d96237ccb490 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:25.135679 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:25.140368 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55d243-85d0-4b79-901a-5c9cf5666c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:25.141140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d243-85d0-4b79-901a-5c9cf5666c44 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:25.141377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d243-85d0-4b79-901a-5c9cf5666c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:25.141869 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d243-85d0-4b79-901a-5c9cf5666c44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:25.142471 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55d243-85d0-4b79-901a-5c9cf5666c44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:25.144457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55d243-85d0-4b79-901a-5c9cf5666c44 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:25.302827 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1051/2076] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:33:25 2022] POST /identity/v3/projects => generated 393 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:25.307264 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3ebc4-77d4-41a0-80ee-aebae06191e8 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:25.319249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3ebc4-77d4-41a0-80ee-aebae06191e8 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:25.388656 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286a4c6-77df-4889-9066-53e07ed4e8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:25.389708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286a4c6-77df-4889-9066-53e07ed4e8ed admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:25.390196 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286a4c6-77df-4889-9066-53e07ed4e8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:25.390668 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286a4c6-77df-4889-9066-53e07ed4e8ed admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:25.391816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286a4c6-77df-4889-9066-53e07ed4e8ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:25.394532 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286a4c6-77df-4889-9066-53e07ed4e8ed admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:25.489917 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1026/2077] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:33:25 2022] POST /identity/v3/users => generated 303 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:25.494278 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55d243-85d0-4b79-901a-5c9cf5666c44 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:25.505646 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55d243-85d0-4b79-901a-5c9cf5666c44 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:25.510542 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f596740-97ff-4e00-ba33-523b2779aae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:25.511060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f596740-97ff-4e00-ba33-523b2779aae7 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:25.516646 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f596740-97ff-4e00-ba33-523b2779aae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:25.516954 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f596740-97ff-4e00-ba33-523b2779aae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:25.519324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f596740-97ff-4e00-ba33-523b2779aae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:25.519439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f596740-97ff-4e00-ba33-523b2779aae7 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:25.532397 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1052/2078] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:25.536752 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286a4c6-77df-4889-9066-53e07ed4e8ed admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:25.547471 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286a4c6-77df-4889-9066-53e07ed4e8ed admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:25.552233 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59a893-fe76-42f8-afe6-d0c35c616424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:25.553134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59a893-fe76-42f8-afe6-d0c35c616424 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:25.553418 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59a893-fe76-42f8-afe6-d0c35c616424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:25.553699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59a893-fe76-42f8-afe6-d0c35c616424 admin admin] PATH_INFO: `/v3/projects/c2b8698d8a004877b7535fbce17bd7ff/users/88e9dfae4240454381e870dd303d149b/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:25.571884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59a893-fe76-42f8-afe6-d0c35c616424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2b8698d8a004877b7535fbce17bd7ff, user_id=88e9dfae4240454381e870dd303d149b, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:25.574486 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59a893-fe76-42f8-afe6-d0c35c616424 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:25.713736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1027/2079] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:25 2022] PUT /identity/v3/projects/c2b8698d8a004877b7535fbce17bd7ff/users/88e9dfae4240454381e870dd303d149b/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:25.718167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f596740-97ff-4e00-ba33-523b2779aae7 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:25.728633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f596740-97ff-4e00-ba33-523b2779aae7 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:25.765936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:25.791444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:25.793788 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70276e95-5c9a-442f-8335-806b0994a023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:25.794531 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70276e95-5c9a-442f-8335-806b0994a023 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:25.794884 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70276e95-5c9a-442f-8335-806b0994a023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:25.795217 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70276e95-5c9a-442f-8335-806b0994a023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:25.795982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70276e95-5c9a-442f-8335-806b0994a023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:25.797906 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70276e95-5c9a-442f-8335-806b0994a023 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:25.808730 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1053/2080] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:25.812717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59a893-fe76-42f8-afe6-d0c35c616424 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:25.827856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59a893-fe76-42f8-afe6-d0c35c616424 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:25.832964 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d5901-a02a-4555-b5c7-472f7e8c438f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:25.833856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d5901-a02a-4555-b5c7-472f7e8c438f admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:25.834146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d5901-a02a-4555-b5c7-472f7e8c438f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:25.834714 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d5901-a02a-4555-b5c7-472f7e8c438f admin admin] PATH_INFO: `/v3/projects/c2b8698d8a004877b7535fbce17bd7ff/users/88e9dfae4240454381e870dd303d149b/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:25.844623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d5901-a02a-4555-b5c7-472f7e8c438f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2b8698d8a004877b7535fbce17bd7ff, user_id=88e9dfae4240454381e870dd303d149b, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:25.844623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d5901-a02a-4555-b5c7-472f7e8c438f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:25.852027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1028/2081] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:25 2022] PUT /identity/v3/projects/c2b8698d8a004877b7535fbce17bd7ff/users/88e9dfae4240454381e870dd303d149b/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:33:25.863161 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf13fab-d3a4-40d7-832a-dbeacefad06d None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:25.863501 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf13fab-d3a4-40d7-832a-dbeacefad06d None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:25.863895 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf13fab-d3a4-40d7-832a-dbeacefad06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:25.895201 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-edf13fab-d3a4-40d7-832a-dbeacefad06d None None] MFA Rules not processed for user `88e9dfae4240454381e870dd303d149b`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:25.896429 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-edf13fab-d3a4-40d7-832a-dbeacefad06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:25.972259 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1054/2082] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:26.022507 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d5901-a02a-4555-b5c7-472f7e8c438f admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:26.024265 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-aa4d5901-a02a-4555-b5c7-472f7e8c438f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:26.089267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d5901-a02a-4555-b5c7-472f7e8c438f admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:26.093384 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973fd36-098c-4b85-9d52-8c83faa948d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:26.094046 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973fd36-098c-4b85-9d52-8c83faa948d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:26.094353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973fd36-098c-4b85-9d52-8c83faa948d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:26.094636 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973fd36-098c-4b85-9d52-8c83faa948d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:26.111135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973fd36-098c-4b85-9d52-8c83faa948d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:26.112849 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973fd36-098c-4b85-9d52-8c83faa948d7 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:26.123427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1029/2083] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:26.179584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:26.240676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf13fab-d3a4-40d7-832a-dbeacefad06d None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:26.250822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf13fab-d3a4-40d7-832a-dbeacefad06d None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:26.254578 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18941e1f-8989-4775-af53-773b675ea28c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:26.255268 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18941e1f-8989-4775-af53-773b675ea28c service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:26.255557 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18941e1f-8989-4775-af53-773b675ea28c service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:26.255832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18941e1f-8989-4775-af53-773b675ea28c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:26.321614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18941e1f-8989-4775-af53-773b675ea28c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:26.323365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18941e1f-8989-4775-af53-773b675ea28c service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:26.334140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1055/2084] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:31.218943 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973fd36-098c-4b85-9d52-8c83faa948d7 service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:31.227985 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973fd36-098c-4b85-9d52-8c83faa948d7 service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:31.232139 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c47ea-e486-45f4-8b4b-07441e60425c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:31.232776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c47ea-e486-45f4-8b4b-07441e60425c service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:31.233060 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c47ea-e486-45f4-8b4b-07441e60425c service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:31.233450 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c47ea-e486-45f4-8b4b-07441e60425c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:31.235498 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-840c47ea-e486-45f4-8b4b-07441e60425c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:31.275837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:31.308337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c47ea-e486-45f4-8b4b-07441e60425c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:31.310309 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c47ea-e486-45f4-8b4b-07441e60425c service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:31.320544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1030/2085] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:33.476922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18941e1f-8989-4775-af53-773b675ea28c service cinder] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:33.478361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-18941e1f-8989-4775-af53-773b675ea28c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:33.494384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18941e1f-8989-4775-af53-773b675ea28c service cinder] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:33.498714 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756c910-ab9e-44bb-b46f-5f7067b5bfa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:33.499393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756c910-ab9e-44bb-b46f-5f7067b5bfa9 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:33.499666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756c910-ab9e-44bb-b46f-5f7067b5bfa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:33.499943 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756c910-ab9e-44bb-b46f-5f7067b5bfa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:33.501773 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-3756c910-ab9e-44bb-b46f-5f7067b5bfa9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:33.555624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:33.569707 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756c910-ab9e-44bb-b46f-5f7067b5bfa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:33.571201 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756c910-ab9e-44bb-b46f-5f7067b5bfa9 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:33.580853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1056/2086] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:41.410005 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=13 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:42.841423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c47ea-e486-45f4-8b4b-07441e60425c service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:42.906822 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c47ea-e486-45f4-8b4b-07441e60425c service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:42.911710 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe7091-36ef-4009-a1b6-80f0b6ad1791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:42.912530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe7091-36ef-4009-a1b6-80f0b6ad1791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:42.912811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe7091-36ef-4009-a1b6-80f0b6ad1791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:42.913086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe7091-36ef-4009-a1b6-80f0b6ad1791 admin admin] PATH_INFO: `/v3/users/69818cb63a374ee8b277790294e70ef9` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:42.914585 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe7091-36ef-4009-a1b6-80f0b6ad1791 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69818cb63a374ee8b277790294e70ef9)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:42.916496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe7091-36ef-4009-a1b6-80f0b6ad1791 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:43.146793 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d4fe7091-36ef-4009-a1b6-80f0b6ad1791 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69818cb63a374ee8b277790294e70ef9'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:43.221850 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d4fe7091-36ef-4009-a1b6-80f0b6ad1791 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69818cb63a374ee8b277790294e70ef9'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:43.222522 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d4fe7091-36ef-4009-a1b6-80f0b6ad1791 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69818cb63a374ee8b277790294e70ef9'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:43.238446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d4fe7091-36ef-4009-a1b6-80f0b6ad1791 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69818cb63a374ee8b277790294e70ef9'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:43.245775 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d4fe7091-36ef-4009-a1b6-80f0b6ad1791 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69818cb63a374ee8b277790294e70ef9'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:43.253463 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1031/2087] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:33:42 2022] DELETE /identity/v3/users/69818cb63a374ee8b277790294e70ef9 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:43.269727 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8f512-8530-41ee-bc40-226255c5b882 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:43.270227 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8f512-8530-41ee-bc40-226255c5b882 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:43.270410 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8f512-8530-41ee-bc40-226255c5b882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:43.288437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-97a8f512-8530-41ee-bc40-226255c5b882 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:43.289689 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-97a8f512-8530-41ee-bc40-226255c5b882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:43.355865 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1057/2088] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:43.367577 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe7091-36ef-4009-a1b6-80f0b6ad1791 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:43.369134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d4fe7091-36ef-4009-a1b6-80f0b6ad1791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:43.429706 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe7091-36ef-4009-a1b6-80f0b6ad1791 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:43.433292 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18b2e3-e5d7-4c1f-a481-d2b9d49ae6d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:43.433969 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18b2e3-e5d7-4c1f-a481-d2b9d49ae6d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:43.434248 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18b2e3-e5d7-4c1f-a481-d2b9d49ae6d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:43.434551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18b2e3-e5d7-4c1f-a481-d2b9d49ae6d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:43.444477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18b2e3-e5d7-4c1f-a481-d2b9d49ae6d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:43.445840 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18b2e3-e5d7-4c1f-a481-d2b9d49ae6d0 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:43.455385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1032/2089] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:44.018707 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8f512-8530-41ee-bc40-226255c5b882 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:44.020500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-97a8f512-8530-41ee-bc40-226255c5b882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:44.034002 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8f512-8530-41ee-bc40-226255c5b882 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:44.038895 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:44.039773 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:44.040052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:44.040325 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf admin admin] PATH_INFO: `/v3/projects/94739478923d405d920943480fb53e29` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:44.041546 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=94739478923d405d920943480fb53e29)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:44.043466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:44.179879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94739478923d405d920943480fb53e29', 'request_id': 'req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:44.253137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94739478923d405d920943480fb53e29', 'request_id': 'req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:44.260898 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94739478923d405d920943480fb53e29', 'request_id': 'req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:44.274055 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf admin admin] The token cache is being invalidate because project 94739478923d405d920943480fb53e29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:33:44.274907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:44.275061 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:44.276723 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1058/2090] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:33:44 2022] DELETE /identity/v3/projects/94739478923d405d920943480fb53e29 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:44.285637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18b2e3-e5d7-4c1f-a481-d2b9d49ae6d0 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:44.296044 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18b2e3-e5d7-4c1f-a481-d2b9d49ae6d0 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:44.317224 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:44.368988 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73658e69-e627-43e1-86a7-5434e381e3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:44.369961 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73658e69-e627-43e1-86a7-5434e381e3a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:44.370397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73658e69-e627-43e1-86a7-5434e381e3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:44.370748 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73658e69-e627-43e1-86a7-5434e381e3a5 admin admin] PATH_INFO: `/v3/users/88e9dfae4240454381e870dd303d149b` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:44.372130 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73658e69-e627-43e1-86a7-5434e381e3a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88e9dfae4240454381e870dd303d149b)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:44.374970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73658e69-e627-43e1-86a7-5434e381e3a5 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:44.651007 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-73658e69-e627-43e1-86a7-5434e381e3a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88e9dfae4240454381e870dd303d149b'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:44.844277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-73658e69-e627-43e1-86a7-5434e381e3a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88e9dfae4240454381e870dd303d149b'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:44.844911 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-73658e69-e627-43e1-86a7-5434e381e3a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88e9dfae4240454381e870dd303d149b'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:44.859425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-73658e69-e627-43e1-86a7-5434e381e3a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88e9dfae4240454381e870dd303d149b'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:44.866376 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-73658e69-e627-43e1-86a7-5434e381e3a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88e9dfae4240454381e870dd303d149b'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:44.872300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1033/2091] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:33:44 2022] DELETE /identity/v3/users/88e9dfae4240454381e870dd303d149b => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:33:45.336933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:45.338755 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:45.354021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ed509-85e1-44cf-ada7-e5435c1afbaf admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:45.413335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f976b193-4fd2-44c9-9e1a-7e3dfb7d243e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:45.414199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976b193-4fd2-44c9-9e1a-7e3dfb7d243e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:45.414514 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976b193-4fd2-44c9-9e1a-7e3dfb7d243e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:45.414794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976b193-4fd2-44c9-9e1a-7e3dfb7d243e admin admin] PATH_INFO: `/v3/projects/c2b8698d8a004877b7535fbce17bd7ff` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:45.415982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f976b193-4fd2-44c9-9e1a-7e3dfb7d243e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2b8698d8a004877b7535fbce17bd7ff)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:45.417917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f976b193-4fd2-44c9-9e1a-7e3dfb7d243e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:45.429241 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:45.541689 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f976b193-4fd2-44c9-9e1a-7e3dfb7d243e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2b8698d8a004877b7535fbce17bd7ff', 'request_id': 'req-f976b193-4fd2-44c9-9e1a-7e3dfb7d243e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:45.649941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f976b193-4fd2-44c9-9e1a-7e3dfb7d243e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2b8698d8a004877b7535fbce17bd7ff', 'request_id': 'req-f976b193-4fd2-44c9-9e1a-7e3dfb7d243e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:45.658326 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f976b193-4fd2-44c9-9e1a-7e3dfb7d243e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2b8698d8a004877b7535fbce17bd7ff', 'request_id': 'req-f976b193-4fd2-44c9-9e1a-7e3dfb7d243e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:45.668717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f976b193-4fd2-44c9-9e1a-7e3dfb7d243e admin admin] The token cache is being invalidate because project c2b8698d8a004877b7535fbce17bd7ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:33:45.669003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f976b193-4fd2-44c9-9e1a-7e3dfb7d243e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:45.669710 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f976b193-4fd2-44c9-9e1a-7e3dfb7d243e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:45.671324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1059/2092] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:33:45 2022] DELETE /identity/v3/projects/c2b8698d8a004877b7535fbce17bd7ff => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:45.676178 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cd85e-7c95-4d81-8d5a-463118bbf82f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:45.676743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cd85e-7c95-4d81-8d5a-463118bbf82f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:45.676915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cd85e-7c95-4d81-8d5a-463118bbf82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:45.698117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-4d7cd85e-7c95-4d81-8d5a-463118bbf82f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:45.699360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4d7cd85e-7c95-4d81-8d5a-463118bbf82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:45.761532 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1034/2093] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:45.770306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9f20e-ec8b-47e8-b3df-34b422fa2d88 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:45.770576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9f20e-ec8b-47e8-b3df-34b422fa2d88 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:45.770860 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9f20e-ec8b-47e8-b3df-34b422fa2d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:45.789793 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6fd9f20e-ec8b-47e8-b3df-34b422fa2d88 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:45.791059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6fd9f20e-ec8b-47e8-b3df-34b422fa2d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:45.799145 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1060/2094] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:45.803508 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7cd85e-7c95-4d81-8d5a-463118bbf82f None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:45.813911 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7cd85e-7c95-4d81-8d5a-463118bbf82f None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:45.818454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef84f2-363b-4693-9b87-e7fb10b29a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:45.821834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef84f2-363b-4693-9b87-e7fb10b29a15 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:45.821834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef84f2-363b-4693-9b87-e7fb10b29a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:45.821834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef84f2-363b-4693-9b87-e7fb10b29a15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:45.821834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef84f2-363b-4693-9b87-e7fb10b29a15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:33:45.821834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef84f2-363b-4693-9b87-e7fb10b29a15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:45.822246 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef84f2-363b-4693-9b87-e7fb10b29a15 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:45.834520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1035/2095] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:33:45.839400 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9f20e-ec8b-47e8-b3df-34b422fa2d88 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:45.848690 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9f20e-ec8b-47e8-b3df-34b422fa2d88 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:45.853748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b8662-fb3a-403b-8ad4-3ce1612b0aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:45.854462 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b8662-fb3a-403b-8ad4-3ce1612b0aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:45.854751 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b8662-fb3a-403b-8ad4-3ce1612b0aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:45.855032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b8662-fb3a-403b-8ad4-3ce1612b0aa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:45.855734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b8662-fb3a-403b-8ad4-3ce1612b0aa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:45.857755 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b8662-fb3a-403b-8ad4-3ce1612b0aa3 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:45.975392 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1061/2096] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:33:45 2022] POST /identity/v3/projects => generated 381 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:45.976436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef84f2-363b-4693-9b87-e7fb10b29a15 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:45.988178 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef84f2-363b-4693-9b87-e7fb10b29a15 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:46.020023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:46.055688 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018d5a9-5a1f-46bf-bb25-af4f1bd3d99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:46.056374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018d5a9-5a1f-46bf-bb25-af4f1bd3d99c admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:46.056658 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018d5a9-5a1f-46bf-bb25-af4f1bd3d99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:46.056932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018d5a9-5a1f-46bf-bb25-af4f1bd3d99c admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:46.057659 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018d5a9-5a1f-46bf-bb25-af4f1bd3d99c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:46.059636 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018d5a9-5a1f-46bf-bb25-af4f1bd3d99c admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:46.205404 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1036/2097] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:33:45 2022] POST /identity/v3/users => generated 291 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:33:46.211925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b8662-fb3a-403b-8ad4-3ce1612b0aa3 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:46.219381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b8662-fb3a-403b-8ad4-3ce1612b0aa3 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:46.224005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08733cca-7818-4ae8-b8a9-630f1967fad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:46.224600 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08733cca-7818-4ae8-b8a9-630f1967fad4 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:46.224882 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08733cca-7818-4ae8-b8a9-630f1967fad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:46.225163 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08733cca-7818-4ae8-b8a9-630f1967fad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:46.225869 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08733cca-7818-4ae8-b8a9-630f1967fad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:46.228528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08733cca-7818-4ae8-b8a9-630f1967fad4 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:46.246319 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1062/2098] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:46.246416 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018d5a9-5a1f-46bf-bb25-af4f1bd3d99c admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:46.257061 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018d5a9-5a1f-46bf-bb25-af4f1bd3d99c admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:46.262466 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d42b97-4572-45ea-b75f-f324b8fee209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:46.263103 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d42b97-4572-45ea-b75f-f324b8fee209 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:46.263427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d42b97-4572-45ea-b75f-f324b8fee209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:46.263947 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d42b97-4572-45ea-b75f-f324b8fee209 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:46.264733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d42b97-4572-45ea-b75f-f324b8fee209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:46.266452 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d42b97-4572-45ea-b75f-f324b8fee209 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:46.277078 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1037/2099] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:33:46.282752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08733cca-7818-4ae8-b8a9-630f1967fad4 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:46.298494 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08733cca-7818-4ae8-b8a9-630f1967fad4 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:46.303091 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd5505-e904-42f8-815b-59582c531847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:46.309729 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd5505-e904-42f8-815b-59582c531847 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:46.309729 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd5505-e904-42f8-815b-59582c531847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:46.309729 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd5505-e904-42f8-815b-59582c531847 admin admin] PATH_INFO: `/v3/projects/501712fc58244162a6a46666916e8ef9/users/9796aa7751cf4fb0994bec016aacb9ff/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:46.319455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd5505-e904-42f8-815b-59582c531847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=501712fc58244162a6a46666916e8ef9, user_id=9796aa7751cf4fb0994bec016aacb9ff, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:46.322104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd5505-e904-42f8-815b-59582c531847 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:46.400009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1063/2100] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:46 2022] PUT /identity/v3/projects/501712fc58244162a6a46666916e8ef9/users/9796aa7751cf4fb0994bec016aacb9ff/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:46.411216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816fa35-c21b-420f-9c65-0b292a62048c None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:46.411868 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816fa35-c21b-420f-9c65-0b292a62048c None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:46.412339 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816fa35-c21b-420f-9c65-0b292a62048c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:46.442162 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-0816fa35-c21b-420f-9c65-0b292a62048c None None] MFA Rules not processed for user `9796aa7751cf4fb0994bec016aacb9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:46.443413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0816fa35-c21b-420f-9c65-0b292a62048c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:46.508662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1038/2101] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:46.514042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd5505-e904-42f8-815b-59582c531847 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:46.589544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd5505-e904-42f8-815b-59582c531847 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:46.594697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdf1a7-befc-4298-9d39-0e66f87c89d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:46.595359 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdf1a7-befc-4298-9d39-0e66f87c89d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:46.595640 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdf1a7-befc-4298-9d39-0e66f87c89d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:46.595915 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdf1a7-befc-4298-9d39-0e66f87c89d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:46.596605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdf1a7-befc-4298-9d39-0e66f87c89d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:46.598523 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdf1a7-befc-4298-9d39-0e66f87c89d4 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:46.746843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1064/2102] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:33:46 2022] POST /identity/v3/projects => generated 383 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:46.751178 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816fa35-c21b-420f-9c65-0b292a62048c None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:46.761932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816fa35-c21b-420f-9c65-0b292a62048c None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:46.819641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234c7a0-02f4-42cf-a370-22bf11291e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:46.820482 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234c7a0-02f4-42cf-a370-22bf11291e8a admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:46.820735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234c7a0-02f4-42cf-a370-22bf11291e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:46.821011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234c7a0-02f4-42cf-a370-22bf11291e8a admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:46.821700 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234c7a0-02f4-42cf-a370-22bf11291e8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:46.823873 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234c7a0-02f4-42cf-a370-22bf11291e8a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:46.919376 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1039/2103] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:33:46 2022] POST /identity/v3/users => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:46.923623 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdf1a7-befc-4298-9d39-0e66f87c89d4 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:46.934697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdf1a7-befc-4298-9d39-0e66f87c89d4 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:46.939720 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e894e-d877-4fb2-9a6a-5807920900b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:46.940380 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e894e-d877-4fb2-9a6a-5807920900b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:46.940665 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e894e-d877-4fb2-9a6a-5807920900b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:46.940943 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e894e-d877-4fb2-9a6a-5807920900b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:46.941669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e894e-d877-4fb2-9a6a-5807920900b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:46.943323 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e894e-d877-4fb2-9a6a-5807920900b7 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:46.954939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1065/2104] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:33:46.959330 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234c7a0-02f4-42cf-a370-22bf11291e8a admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:46.970947 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234c7a0-02f4-42cf-a370-22bf11291e8a admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:46.975877 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7112d9-2856-41bd-a55c-8a73d1c400da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:46.976784 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7112d9-2856-41bd-a55c-8a73d1c400da admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:46.977065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7112d9-2856-41bd-a55c-8a73d1c400da admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:46.977344 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7112d9-2856-41bd-a55c-8a73d1c400da admin admin] PATH_INFO: `/v3/projects/7269ecdee09c4c74b9b5eb56a3194a15/users/3ab902a9720f49db945ca5d6a4c1ea6f/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:46.993268 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7112d9-2856-41bd-a55c-8a73d1c400da admin admin] RBAC: Authorizing `identity:create_grant(project_id=7269ecdee09c4c74b9b5eb56a3194a15, user_id=3ab902a9720f49db945ca5d6a4c1ea6f, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:46.996028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7112d9-2856-41bd-a55c-8a73d1c400da admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:47.140204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1040/2105] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:46 2022] PUT /identity/v3/projects/7269ecdee09c4c74b9b5eb56a3194a15/users/3ab902a9720f49db945ca5d6a4c1ea6f/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:47.145743 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e894e-d877-4fb2-9a6a-5807920900b7 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:47.157026 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e894e-d877-4fb2-9a6a-5807920900b7 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:47.166248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:47.243998 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24872095-0c35-49e5-9172-4aa321e2a96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:47.246170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24872095-0c35-49e5-9172-4aa321e2a96a admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:47.246170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24872095-0c35-49e5-9172-4aa321e2a96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:47.246170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24872095-0c35-49e5-9172-4aa321e2a96a admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:47.246431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24872095-0c35-49e5-9172-4aa321e2a96a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:47.248528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24872095-0c35-49e5-9172-4aa321e2a96a admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:47.259121 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1066/2106] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:47.263778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7112d9-2856-41bd-a55c-8a73d1c400da admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:47.269472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:47.275349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7112d9-2856-41bd-a55c-8a73d1c400da admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:47.280458 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd185fd-64fb-435a-a377-0c3c029e013f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:47.281383 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd185fd-64fb-435a-a377-0c3c029e013f admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:47.281668 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd185fd-64fb-435a-a377-0c3c029e013f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:47.281977 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd185fd-64fb-435a-a377-0c3c029e013f admin admin] PATH_INFO: `/v3/projects/7269ecdee09c4c74b9b5eb56a3194a15/users/3ab902a9720f49db945ca5d6a4c1ea6f/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:47.284044 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd185fd-64fb-435a-a377-0c3c029e013f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7269ecdee09c4c74b9b5eb56a3194a15, user_id=3ab902a9720f49db945ca5d6a4c1ea6f, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:47.286772 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd185fd-64fb-435a-a377-0c3c029e013f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:47.298510 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1041/2107] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:47 2022] PUT /identity/v3/projects/7269ecdee09c4c74b9b5eb56a3194a15/users/3ab902a9720f49db945ca5d6a4c1ea6f/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:33:47.311328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c617a-9263-48d8-9d4c-b0b03720d181 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:47.311582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c617a-9263-48d8-9d4c-b0b03720d181 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:47.311867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c617a-9263-48d8-9d4c-b0b03720d181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:47.345329 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6f5c617a-9263-48d8-9d4c-b0b03720d181 None None] MFA Rules not processed for user `3ab902a9720f49db945ca5d6a4c1ea6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:47.346655 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6f5c617a-9263-48d8-9d4c-b0b03720d181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:47.414588 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1067/2108] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:33:47.425505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd185fd-64fb-435a-a377-0c3c029e013f admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:47.427217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8dd185fd-64fb-435a-a377-0c3c029e013f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:47.484805 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd185fd-64fb-435a-a377-0c3c029e013f admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:47.489353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5b8db-447d-40aa-8cf9-9cc459415860 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:47.490001 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5b8db-447d-40aa-8cf9-9cc459415860 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:47.490303 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5b8db-447d-40aa-8cf9-9cc459415860 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:47.490580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5b8db-447d-40aa-8cf9-9cc459415860 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:47.500117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5b8db-447d-40aa-8cf9-9cc459415860 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:47.509308 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5b8db-447d-40aa-8cf9-9cc459415860 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:47.511106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1042/2109] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:47.611504 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c617a-9263-48d8-9d4c-b0b03720d181 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:47.621480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c617a-9263-48d8-9d4c-b0b03720d181 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:47.625902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c97c6-04de-4a98-b60d-aed3ceb79163 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:47.626582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c97c6-04de-4a98-b60d-aed3ceb79163 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:47.626861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c97c6-04de-4a98-b60d-aed3ceb79163 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:47.627134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c97c6-04de-4a98-b60d-aed3ceb79163 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:47.690208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c97c6-04de-4a98-b60d-aed3ceb79163 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:47.691732 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c97c6-04de-4a98-b60d-aed3ceb79163 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:47.702276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1068/2110] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:48.298236 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5b8db-447d-40aa-8cf9-9cc459415860 service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:48.299830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-86a5b8db-447d-40aa-8cf9-9cc459415860 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:48.396581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5b8db-447d-40aa-8cf9-9cc459415860 service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:48.405132 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88760ec-c584-4e91-86b6-f735758baaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:48.406674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88760ec-c584-4e91-86b6-f735758baaca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:48.407175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88760ec-c584-4e91-86b6-f735758baaca admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:48.407520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88760ec-c584-4e91-86b6-f735758baaca admin admin] PATH_INFO: `/v3/users/e90624c2891e428cb237b08aa283042e` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:48.408845 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88760ec-c584-4e91-86b6-f735758baaca admin admin] RBAC: Authorizing `identity:delete_user(user_id=e90624c2891e428cb237b08aa283042e)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:48.410993 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88760ec-c584-4e91-86b6-f735758baaca admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:48.630295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e88760ec-c584-4e91-86b6-f735758baaca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e90624c2891e428cb237b08aa283042e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:48.759634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e88760ec-c584-4e91-86b6-f735758baaca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e90624c2891e428cb237b08aa283042e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:48.760668 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e88760ec-c584-4e91-86b6-f735758baaca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e90624c2891e428cb237b08aa283042e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:48.784636 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e88760ec-c584-4e91-86b6-f735758baaca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e90624c2891e428cb237b08aa283042e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:48.812797 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e88760ec-c584-4e91-86b6-f735758baaca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e90624c2891e428cb237b08aa283042e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:48.821176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1043/2111] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:33:48 2022] DELETE /identity/v3/users/e90624c2891e428cb237b08aa283042e => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:48.825794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ed0cb-93ae-40af-9ad7-40c6b0d245ab None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:48.826891 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ed0cb-93ae-40af-9ad7-40c6b0d245ab None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:48.827021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ed0cb-93ae-40af-9ad7-40c6b0d245ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:48.846187 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-e66ed0cb-93ae-40af-9ad7-40c6b0d245ab None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:48.847594 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e66ed0cb-93ae-40af-9ad7-40c6b0d245ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:48.874441 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:48.910298 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1069/2112] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:48.920321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88760ec-c584-4e91-86b6-f735758baaca admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:48.921832 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e88760ec-c584-4e91-86b6-f735758baaca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:48.938537 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:49.025496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88760ec-c584-4e91-86b6-f735758baaca admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:49.030586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91bee9-b2f7-44c4-9864-b4923e4334a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:49.031315 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91bee9-b2f7-44c4-9864-b4923e4334a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:49.031977 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91bee9-b2f7-44c4-9864-b4923e4334a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:49.031977 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91bee9-b2f7-44c4-9864-b4923e4334a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:49.042936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e91bee9-b2f7-44c4-9864-b4923e4334a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:49.044572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e91bee9-b2f7-44c4-9864-b4923e4334a9 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:49.056347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1044/2113] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:49.627196 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ed0cb-93ae-40af-9ad7-40c6b0d245ab None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:49.628631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e66ed0cb-93ae-40af-9ad7-40c6b0d245ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:49.653333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ed0cb-93ae-40af-9ad7-40c6b0d245ab None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:49.659547 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a2a19-9d5e-48e4-9f3f-9ea112f6939e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:49.660442 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a2a19-9d5e-48e4-9f3f-9ea112f6939e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:49.660726 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a2a19-9d5e-48e4-9f3f-9ea112f6939e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:49.661055 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a2a19-9d5e-48e4-9f3f-9ea112f6939e admin admin] PATH_INFO: `/v3/projects/bd3364f2b2fc49788daa0f4b03143015` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:49.668536 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a2a19-9d5e-48e4-9f3f-9ea112f6939e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd3364f2b2fc49788daa0f4b03143015)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:49.668536 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a2a19-9d5e-48e4-9f3f-9ea112f6939e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:49.801955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e30a2a19-9d5e-48e4-9f3f-9ea112f6939e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd3364f2b2fc49788daa0f4b03143015', 'request_id': 'req-e30a2a19-9d5e-48e4-9f3f-9ea112f6939e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:49.890255 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e30a2a19-9d5e-48e4-9f3f-9ea112f6939e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd3364f2b2fc49788daa0f4b03143015', 'request_id': 'req-e30a2a19-9d5e-48e4-9f3f-9ea112f6939e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:49.900193 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e30a2a19-9d5e-48e4-9f3f-9ea112f6939e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd3364f2b2fc49788daa0f4b03143015', 'request_id': 'req-e30a2a19-9d5e-48e4-9f3f-9ea112f6939e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:49.911613 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e30a2a19-9d5e-48e4-9f3f-9ea112f6939e admin admin] The token cache is being invalidate because project bd3364f2b2fc49788daa0f4b03143015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:33:49.911905 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e30a2a19-9d5e-48e4-9f3f-9ea112f6939e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:49.912597 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e30a2a19-9d5e-48e4-9f3f-9ea112f6939e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:49.918452 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1070/2114] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:33:49 2022] DELETE /identity/v3/projects/bd3364f2b2fc49788daa0f4b03143015 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:49.920775 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43008ff-3cc1-443c-8fa7-151046826ef9 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:49.921415 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43008ff-3cc1-443c-8fa7-151046826ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:49.921703 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43008ff-3cc1-443c-8fa7-151046826ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:49.942020 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-d43008ff-3cc1-443c-8fa7-151046826ef9 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:49.943329 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d43008ff-3cc1-443c-8fa7-151046826ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:50.007121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1045/2115] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:50.023902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0334cc-698b-43c1-9af7-07661e7bb851 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:50.024283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0334cc-698b-43c1-9af7-07661e7bb851 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:50.024635 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0334cc-698b-43c1-9af7-07661e7bb851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:50.045723 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-8c0334cc-698b-43c1-9af7-07661e7bb851 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:50.047029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8c0334cc-698b-43c1-9af7-07661e7bb851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:50.055918 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1071/2116] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:50.060862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43008ff-3cc1-443c-8fa7-151046826ef9 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:50.073220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43008ff-3cc1-443c-8fa7-151046826ef9 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:50.080299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb2ce5-cfac-4276-8e07-b093866445da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:50.080970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb2ce5-cfac-4276-8e07-b093866445da admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:50.081252 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb2ce5-cfac-4276-8e07-b093866445da admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:50.081526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb2ce5-cfac-4276-8e07-b093866445da admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:50.083171 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb2ce5-cfac-4276-8e07-b093866445da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:33:50.083171 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb2ce5-cfac-4276-8e07-b093866445da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:50.084300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb2ce5-cfac-4276-8e07-b093866445da admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:50.102316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1046/2117] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:33:50.105197 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0334cc-698b-43c1-9af7-07661e7bb851 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:50.115163 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0334cc-698b-43c1-9af7-07661e7bb851 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:50.120120 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a4f5d-1573-4ca8-9e76-f0f21abfc2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:50.120789 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a4f5d-1573-4ca8-9e76-f0f21abfc2c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:50.121074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a4f5d-1573-4ca8-9e76-f0f21abfc2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:50.121365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a4f5d-1573-4ca8-9e76-f0f21abfc2c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:50.122070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a4f5d-1573-4ca8-9e76-f0f21abfc2c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:50.124568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a4f5d-1573-4ca8-9e76-f0f21abfc2c3 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:50.223017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1072/2118] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:33:50 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:50.233272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb2ce5-cfac-4276-8e07-b093866445da admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:50.263673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb2ce5-cfac-4276-8e07-b093866445da admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:50.352598 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ef302-c9e1-468b-a63b-8a9b877f5549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:50.353589 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ef302-c9e1-468b-a63b-8a9b877f5549 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:50.354243 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ef302-c9e1-468b-a63b-8a9b877f5549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:50.354710 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ef302-c9e1-468b-a63b-8a9b877f5549 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:50.355777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ef302-c9e1-468b-a63b-8a9b877f5549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:50.358593 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ef302-c9e1-468b-a63b-8a9b877f5549 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:50.471891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1047/2119] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:33:50 2022] POST /identity/v3/users => generated 283 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:50.476054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a4f5d-1573-4ca8-9e76-f0f21abfc2c3 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:50.489875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a4f5d-1573-4ca8-9e76-f0f21abfc2c3 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:50.492449 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4fa05-82da-40a3-b9bd-e8de6effcce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:50.493129 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4fa05-82da-40a3-b9bd-e8de6effcce4 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:50.493413 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4fa05-82da-40a3-b9bd-e8de6effcce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:50.493691 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4fa05-82da-40a3-b9bd-e8de6effcce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:50.494928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d4fa05-82da-40a3-b9bd-e8de6effcce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:50.498413 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d4fa05-82da-40a3-b9bd-e8de6effcce4 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:50.511150 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1073/2120] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:50.516940 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ef302-c9e1-468b-a63b-8a9b877f5549 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:50.527466 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ef302-c9e1-468b-a63b-8a9b877f5549 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:50.534575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf2dc1-e919-4762-aa76-9a48d62b8e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:50.535344 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf2dc1-e919-4762-aa76-9a48d62b8e65 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:50.535635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf2dc1-e919-4762-aa76-9a48d62b8e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:50.535921 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf2dc1-e919-4762-aa76-9a48d62b8e65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:50.536673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf2dc1-e919-4762-aa76-9a48d62b8e65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:50.538502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf2dc1-e919-4762-aa76-9a48d62b8e65 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:50.550057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1048/2121] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:50.554601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4fa05-82da-40a3-b9bd-e8de6effcce4 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:50.564780 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4fa05-82da-40a3-b9bd-e8de6effcce4 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:50.571702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85065b9f-8b60-4aa8-af37-e2d510c1ca5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:50.572640 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85065b9f-8b60-4aa8-af37-e2d510c1ca5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:50.572913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85065b9f-8b60-4aa8-af37-e2d510c1ca5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:50.573179 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85065b9f-8b60-4aa8-af37-e2d510c1ca5d admin admin] PATH_INFO: `/v3/projects/f20de04a5f9f4355bbad81e50380a12f/users/d3e4a363addc490ab87a11aa7e158460/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:50.604075 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85065b9f-8b60-4aa8-af37-e2d510c1ca5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20de04a5f9f4355bbad81e50380a12f, user_id=d3e4a363addc490ab87a11aa7e158460, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:50.607454 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85065b9f-8b60-4aa8-af37-e2d510c1ca5d admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:50.725246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1074/2122] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:50 2022] PUT /identity/v3/projects/f20de04a5f9f4355bbad81e50380a12f/users/d3e4a363addc490ab87a11aa7e158460/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:50.737337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110935ed-c1c4-4d92-ab7d-15ae7516946d None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:50.737783 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110935ed-c1c4-4d92-ab7d-15ae7516946d None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:50.738213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110935ed-c1c4-4d92-ab7d-15ae7516946d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:50.791270 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-110935ed-c1c4-4d92-ab7d-15ae7516946d None None] MFA Rules not processed for user `d3e4a363addc490ab87a11aa7e158460`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:50.792629 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-110935ed-c1c4-4d92-ab7d-15ae7516946d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:50.802651 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:50.872218 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1049/2123] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:33:51.334856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85065b9f-8b60-4aa8-af37-e2d510c1ca5d admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:51.336746 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-85065b9f-8b60-4aa8-af37-e2d510c1ca5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:51.337198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110935ed-c1c4-4d92-ab7d-15ae7516946d None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:51.338932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-110935ed-c1c4-4d92-ab7d-15ae7516946d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:51.400073 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:51.413830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85065b9f-8b60-4aa8-af37-e2d510c1ca5d admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:51.418978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e67502-cb26-4861-a1ec-ccab848506a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:51.420338 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e67502-cb26-4861-a1ec-ccab848506a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:51.420617 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e67502-cb26-4861-a1ec-ccab848506a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:51.420903 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e67502-cb26-4861-a1ec-ccab848506a0 admin admin] PATH_INFO: `/v3/users/ccc30a04d1de4114a0f42b90b1f17166` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:51.422733 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e67502-cb26-4861-a1ec-ccab848506a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccc30a04d1de4114a0f42b90b1f17166)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:51.424770 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e67502-cb26-4861-a1ec-ccab848506a0 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:51.443915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110935ed-c1c4-4d92-ab7d-15ae7516946d None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:51.448537 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03524d-d95b-45d5-a46f-b0ee7c354262 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:51.449217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03524d-d95b-45d5-a46f-b0ee7c354262 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:51.449494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03524d-d95b-45d5-a46f-b0ee7c354262 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:51.449811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03524d-d95b-45d5-a46f-b0ee7c354262 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:51.461759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b03524d-d95b-45d5-a46f-b0ee7c354262 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:51.463657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b03524d-d95b-45d5-a46f-b0ee7c354262 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:51.474390 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1050/2124] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:51 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:33:51.607927 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-d0e67502-cb26-4861-a1ec-ccab848506a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccc30a04d1de4114a0f42b90b1f17166'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:51.696036 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-d0e67502-cb26-4861-a1ec-ccab848506a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccc30a04d1de4114a0f42b90b1f17166'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:51.706496 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-d0e67502-cb26-4861-a1ec-ccab848506a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccc30a04d1de4114a0f42b90b1f17166'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:51.707652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-d0e67502-cb26-4861-a1ec-ccab848506a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccc30a04d1de4114a0f42b90b1f17166'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:51.721648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-d0e67502-cb26-4861-a1ec-ccab848506a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccc30a04d1de4114a0f42b90b1f17166'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:51.727956 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1075/2125] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:33:51 2022] DELETE /identity/v3/users/ccc30a04d1de4114a0f42b90b1f17166 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:51.733461 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43c84a-f81f-4141-b215-b3893b18fec6 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:51.733746 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43c84a-f81f-4141-b215-b3893b18fec6 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:51.734019 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43c84a-f81f-4141-b215-b3893b18fec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:51.751903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-4b43c84a-f81f-4141-b215-b3893b18fec6 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:51.753118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4b43c84a-f81f-4141-b215-b3893b18fec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:51.824906 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1051/2126] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:51.836003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e67502-cb26-4861-a1ec-ccab848506a0 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:51.839212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d0e67502-cb26-4861-a1ec-ccab848506a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:51.906491 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e67502-cb26-4861-a1ec-ccab848506a0 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:51.910126 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f52ee-4342-474f-9ede-46aee3c21b21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:51.910931 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f52ee-4342-474f-9ede-46aee3c21b21 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:51.911222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f52ee-4342-474f-9ede-46aee3c21b21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:51.911633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f52ee-4342-474f-9ede-46aee3c21b21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:51.923199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f52ee-4342-474f-9ede-46aee3c21b21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:51.926222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f52ee-4342-474f-9ede-46aee3c21b21 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:51.935704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1076/2127] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:52.302940 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43c84a-f81f-4141-b215-b3893b18fec6 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:52.305577 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4b43c84a-f81f-4141-b215-b3893b18fec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:52.348779 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43c84a-f81f-4141-b215-b3893b18fec6 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:52.359711 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b19bf5-31ec-438b-afea-5c6244341439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:52.360609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b19bf5-31ec-438b-afea-5c6244341439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:52.360893 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b19bf5-31ec-438b-afea-5c6244341439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:52.361172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b19bf5-31ec-438b-afea-5c6244341439 admin admin] PATH_INFO: `/v3/users/9796aa7751cf4fb0994bec016aacb9ff` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:52.362516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b19bf5-31ec-438b-afea-5c6244341439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9796aa7751cf4fb0994bec016aacb9ff)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:52.364447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b19bf5-31ec-438b-afea-5c6244341439 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:52.550711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f52ee-4342-474f-9ede-46aee3c21b21 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:52.553830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2d4f52ee-4342-474f-9ede-46aee3c21b21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:52.574043 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f52ee-4342-474f-9ede-46aee3c21b21 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:52.579732 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721591ba-cc48-4143-8e31-f239e2c19933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:52.581344 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721591ba-cc48-4143-8e31-f239e2c19933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:52.581635 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721591ba-cc48-4143-8e31-f239e2c19933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:52.582110 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721591ba-cc48-4143-8e31-f239e2c19933 admin admin] PATH_INFO: `/v3/projects/29937f726169487981f22df2eadf07f4` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:52.583722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721591ba-cc48-4143-8e31-f239e2c19933 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29937f726169487981f22df2eadf07f4)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:52.585734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721591ba-cc48-4143-8e31-f239e2c19933 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:52.621909 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e6b19bf5-31ec-438b-afea-5c6244341439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9796aa7751cf4fb0994bec016aacb9ff'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:52.624258 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:52.725808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-721591ba-cc48-4143-8e31-f239e2c19933 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29937f726169487981f22df2eadf07f4', 'request_id': 'req-721591ba-cc48-4143-8e31-f239e2c19933'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:52.769471 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e6b19bf5-31ec-438b-afea-5c6244341439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9796aa7751cf4fb0994bec016aacb9ff'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:52.770104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e6b19bf5-31ec-438b-afea-5c6244341439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9796aa7751cf4fb0994bec016aacb9ff'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:52.782910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e6b19bf5-31ec-438b-afea-5c6244341439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9796aa7751cf4fb0994bec016aacb9ff'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:52.790778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e6b19bf5-31ec-438b-afea-5c6244341439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9796aa7751cf4fb0994bec016aacb9ff'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:52.798640 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1052/2128] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:33:52 2022] DELETE /identity/v3/users/9796aa7751cf4fb0994bec016aacb9ff => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:33:52.805093 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fc0f6-05a8-48de-bcab-3718ded31b44 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:52.805494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fc0f6-05a8-48de-bcab-3718ded31b44 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:52.805878 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fc0f6-05a8-48de-bcab-3718ded31b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:52.826503 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-987fc0f6-05a8-48de-bcab-3718ded31b44 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:52.827844 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-987fc0f6-05a8-48de-bcab-3718ded31b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:52.907168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1053/2129] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:33:52.909896 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fc0f6-05a8-48de-bcab-3718ded31b44 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:52.911306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-987fc0f6-05a8-48de-bcab-3718ded31b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:52.970685 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fc0f6-05a8-48de-bcab-3718ded31b44 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:52.974465 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e0877-9a08-4ca0-ae86-d27f39ff3169 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:52.975150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e0877-9a08-4ca0-ae86-d27f39ff3169 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:52.975438 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e0877-9a08-4ca0-ae86-d27f39ff3169 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:52.975726 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e0877-9a08-4ca0-ae86-d27f39ff3169 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:52.982792 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-721591ba-cc48-4143-8e31-f239e2c19933 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29937f726169487981f22df2eadf07f4', 'request_id': 'req-721591ba-cc48-4143-8e31-f239e2c19933'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:52.986804 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e0877-9a08-4ca0-ae86-d27f39ff3169 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:52.988355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e0877-9a08-4ca0-ae86-d27f39ff3169 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:52.990970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-721591ba-cc48-4143-8e31-f239e2c19933 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29937f726169487981f22df2eadf07f4', 'request_id': 'req-721591ba-cc48-4143-8e31-f239e2c19933'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:52.996629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:52.998911 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1054/2130] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:33:53.004300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-721591ba-cc48-4143-8e31-f239e2c19933 admin admin] The token cache is being invalidate because project 29937f726169487981f22df2eadf07f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:33:53.004597 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-721591ba-cc48-4143-8e31-f239e2c19933 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:53.005305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-721591ba-cc48-4143-8e31-f239e2c19933 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:53.006949 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1077/2131] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:33:52 2022] DELETE /identity/v3/projects/29937f726169487981f22df2eadf07f4 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:53.011144 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e0877-9a08-4ca0-ae86-d27f39ff3169 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:53.012777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-c53e0877-9a08-4ca0-ae86-d27f39ff3169 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:53.025804 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e0877-9a08-4ca0-ae86-d27f39ff3169 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:53.030903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37d877-959d-4c36-8b29-34bc4be952db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:53.031775 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37d877-959d-4c36-8b29-34bc4be952db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:53.032054 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37d877-959d-4c36-8b29-34bc4be952db admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:53.032460 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37d877-959d-4c36-8b29-34bc4be952db admin admin] PATH_INFO: `/v3/users/ad50c121d1f04c98a680409bdd9b9336` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:53.033706 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37d877-959d-4c36-8b29-34bc4be952db admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad50c121d1f04c98a680409bdd9b9336)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:53.035661 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37d877-959d-4c36-8b29-34bc4be952db admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:53.326153 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-bb37d877-959d-4c36-8b29-34bc4be952db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad50c121d1f04c98a680409bdd9b9336'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:53.430449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-bb37d877-959d-4c36-8b29-34bc4be952db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad50c121d1f04c98a680409bdd9b9336'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:53.431737 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-bb37d877-959d-4c36-8b29-34bc4be952db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad50c121d1f04c98a680409bdd9b9336'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:53.448218 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-bb37d877-959d-4c36-8b29-34bc4be952db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad50c121d1f04c98a680409bdd9b9336'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:53.456765 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-bb37d877-959d-4c36-8b29-34bc4be952db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad50c121d1f04c98a680409bdd9b9336'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:53.466349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1055/2132] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:33:53 2022] DELETE /identity/v3/users/ad50c121d1f04c98a680409bdd9b9336 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:33:53.477403 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721591ba-cc48-4143-8e31-f239e2c19933 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:53.490777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-721591ba-cc48-4143-8e31-f239e2c19933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:53.606139 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721591ba-cc48-4143-8e31-f239e2c19933 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:53.700405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:53.701986 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:53.702405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:53.702708 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b admin admin] PATH_INFO: `/v3/projects/501712fc58244162a6a46666916e8ef9` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:53.704724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=501712fc58244162a6a46666916e8ef9)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:53.707498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:53.889745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '501712fc58244162a6a46666916e8ef9', 'request_id': 'req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:54.035796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '501712fc58244162a6a46666916e8ef9', 'request_id': 'req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:54.047118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '501712fc58244162a6a46666916e8ef9', 'request_id': 'req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:54.058827 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b admin admin] The token cache is being invalidate because project 501712fc58244162a6a46666916e8ef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:33:54.059120 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:54.059785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:54.062200 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1078/2133] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:33:53 2022] DELETE /identity/v3/projects/501712fc58244162a6a46666916e8ef9 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:54.066180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37d877-959d-4c36-8b29-34bc4be952db admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:54.067760 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-bb37d877-959d-4c36-8b29-34bc4be952db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:54.082377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37d877-959d-4c36-8b29-34bc4be952db admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:54.164066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdf670-1953-4f0c-959c-3a3283d97c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:54.165878 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdf670-1953-4f0c-959c-3a3283d97c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:54.166452 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdf670-1953-4f0c-959c-3a3283d97c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:54.166842 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdf670-1953-4f0c-959c-3a3283d97c20 admin admin] PATH_INFO: `/v3/users/3ab902a9720f49db945ca5d6a4c1ea6f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:54.168895 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbdf670-1953-4f0c-959c-3a3283d97c20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ab902a9720f49db945ca5d6a4c1ea6f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:54.172453 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbdf670-1953-4f0c-959c-3a3283d97c20 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:54.310944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:54.375592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:54.377653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:54.403895 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c185b-fe3d-44fc-8e78-30ec792cfd5b admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:54.407621 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-0bbdf670-1953-4f0c-959c-3a3283d97c20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ab902a9720f49db945ca5d6a4c1ea6f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:54.412489 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b34cd-1091-487f-929a-c2f29cff9f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:54.413481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b34cd-1091-487f-929a-c2f29cff9f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:54.413859 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b34cd-1091-487f-929a-c2f29cff9f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:54.414204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b34cd-1091-487f-929a-c2f29cff9f99 admin admin] PATH_INFO: `/v3/projects/b3e3804d06ad496da11844d35def6204` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:54.418847 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b34cd-1091-487f-929a-c2f29cff9f99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3e3804d06ad496da11844d35def6204)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:54.418847 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b34cd-1091-487f-929a-c2f29cff9f99 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:54.496025 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-0bbdf670-1953-4f0c-959c-3a3283d97c20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ab902a9720f49db945ca5d6a4c1ea6f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:54.497020 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-0bbdf670-1953-4f0c-959c-3a3283d97c20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ab902a9720f49db945ca5d6a4c1ea6f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:54.518799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-0bbdf670-1953-4f0c-959c-3a3283d97c20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ab902a9720f49db945ca5d6a4c1ea6f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:54.527975 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-0bbdf670-1953-4f0c-959c-3a3283d97c20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ab902a9720f49db945ca5d6a4c1ea6f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:54.535634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1056/2134] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:33:54 2022] DELETE /identity/v3/users/3ab902a9720f49db945ca5d6a4c1ea6f => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:54.548800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-910b34cd-1091-487f-929a-c2f29cff9f99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3e3804d06ad496da11844d35def6204', 'request_id': 'req-910b34cd-1091-487f-929a-c2f29cff9f99'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:54.658598 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-910b34cd-1091-487f-929a-c2f29cff9f99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3e3804d06ad496da11844d35def6204', 'request_id': 'req-910b34cd-1091-487f-929a-c2f29cff9f99'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:54.665934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-910b34cd-1091-487f-929a-c2f29cff9f99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3e3804d06ad496da11844d35def6204', 'request_id': 'req-910b34cd-1091-487f-929a-c2f29cff9f99'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:54.678235 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-910b34cd-1091-487f-929a-c2f29cff9f99 admin admin] The token cache is being invalidate because project b3e3804d06ad496da11844d35def6204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:33:54.678616 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-910b34cd-1091-487f-929a-c2f29cff9f99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:54.679340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-910b34cd-1091-487f-929a-c2f29cff9f99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:54.680990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1079/2135] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:33:54 2022] DELETE /identity/v3/projects/b3e3804d06ad496da11844d35def6204 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:54.687234 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74e1e6-a5b7-47da-b54c-aeff765a48fb None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:54.687544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74e1e6-a5b7-47da-b54c-aeff765a48fb None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:54.688135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74e1e6-a5b7-47da-b54c-aeff765a48fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:54.708750 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-dc74e1e6-a5b7-47da-b54c-aeff765a48fb None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:54.710119 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-dc74e1e6-a5b7-47da-b54c-aeff765a48fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:54.832274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1057/2136] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:33:54.843802 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4bba0-174a-4450-891a-0120b42ae379 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:54.844081 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4bba0-174a-4450-891a-0120b42ae379 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:54.844520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4bba0-174a-4450-891a-0120b42ae379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:54.869536 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-08d4bba0-174a-4450-891a-0120b42ae379 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:54.871337 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-08d4bba0-174a-4450-891a-0120b42ae379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:54.879625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1080/2137] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:54.885802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74e1e6-a5b7-47da-b54c-aeff765a48fb None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:54.896473 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74e1e6-a5b7-47da-b54c-aeff765a48fb None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:54.901465 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14853e28-0314-4722-9b14-d3b5c4e8c91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:54.902316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14853e28-0314-4722-9b14-d3b5c4e8c91d admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:54.902637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14853e28-0314-4722-9b14-d3b5c4e8c91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:54.903119 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14853e28-0314-4722-9b14-d3b5c4e8c91d admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:54.904400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14853e28-0314-4722-9b14-d3b5c4e8c91d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:33:54.904707 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14853e28-0314-4722-9b14-d3b5c4e8c91d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:54.906517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14853e28-0314-4722-9b14-d3b5c4e8c91d admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:54.940150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1058/2138] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:33:54.941616 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4bba0-174a-4450-891a-0120b42ae379 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:54.964831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4bba0-174a-4450-891a-0120b42ae379 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:54.975042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90904e9c-672c-4877-afe8-3a8fc23768bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:54.976474 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90904e9c-672c-4877-afe8-3a8fc23768bd admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:54.977213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90904e9c-672c-4877-afe8-3a8fc23768bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:54.977882 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90904e9c-672c-4877-afe8-3a8fc23768bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:54.979487 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90904e9c-672c-4877-afe8-3a8fc23768bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:54.984834 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90904e9c-672c-4877-afe8-3a8fc23768bd admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:55.030315 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14853e28-0314-4722-9b14-d3b5c4e8c91d admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:55.035540 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-14853e28-0314-4722-9b14-d3b5c4e8c91d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:55.057349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14853e28-0314-4722-9b14-d3b5c4e8c91d admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:55.062099 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31f08a-ae4c-4afa-9b0a-b802d40c5734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:55.063115 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31f08a-ae4c-4afa-9b0a-b802d40c5734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:55.063475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31f08a-ae4c-4afa-9b0a-b802d40c5734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:55.063824 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31f08a-ae4c-4afa-9b0a-b802d40c5734 admin admin] PATH_INFO: `/v3/projects/7269ecdee09c4c74b9b5eb56a3194a15` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:55.065407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31f08a-ae4c-4afa-9b0a-b802d40c5734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7269ecdee09c4c74b9b5eb56a3194a15)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:55.067852 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31f08a-ae4c-4afa-9b0a-b802d40c5734 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:55.132922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1081/2139] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:33:54 2022] POST /identity/v3/projects => generated 369 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:33:55.137441 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90904e9c-672c-4877-afe8-3a8fc23768bd admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:55.155332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:55.225030 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90904e9c-672c-4877-afe8-3a8fc23768bd admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:55.237200 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5d31f08a-ae4c-4afa-9b0a-b802d40c5734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7269ecdee09c4c74b9b5eb56a3194a15', 'request_id': 'req-5d31f08a-ae4c-4afa-9b0a-b802d40c5734'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:55.285424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7dddb2-7f31-4353-ac83-612b22d465c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:55.286123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dddb2-7f31-4353-ac83-612b22d465c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:55.288500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dddb2-7f31-4353-ac83-612b22d465c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:55.288500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dddb2-7f31-4353-ac83-612b22d465c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:55.288500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7dddb2-7f31-4353-ac83-612b22d465c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:55.289360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7dddb2-7f31-4353-ac83-612b22d465c2 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:55.339420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5d31f08a-ae4c-4afa-9b0a-b802d40c5734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7269ecdee09c4c74b9b5eb56a3194a15', 'request_id': 'req-5d31f08a-ae4c-4afa-9b0a-b802d40c5734'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:55.350292 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5d31f08a-ae4c-4afa-9b0a-b802d40c5734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7269ecdee09c4c74b9b5eb56a3194a15', 'request_id': 'req-5d31f08a-ae4c-4afa-9b0a-b802d40c5734'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:55.368763 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5d31f08a-ae4c-4afa-9b0a-b802d40c5734 admin admin] The token cache is being invalidate because project 7269ecdee09c4c74b9b5eb56a3194a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:33:55.371083 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5d31f08a-ae4c-4afa-9b0a-b802d40c5734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:55.371083 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5d31f08a-ae4c-4afa-9b0a-b802d40c5734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:33:55.371583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1059/2140] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:33:55 2022] DELETE /identity/v3/projects/7269ecdee09c4c74b9b5eb56a3194a15 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:33:55.376449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde667bf-86c4-4131-933f-076c2a59dc3a None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:55.376738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde667bf-86c4-4131-933f-076c2a59dc3a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:55.377080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde667bf-86c4-4131-933f-076c2a59dc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:55.396563 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-bde667bf-86c4-4131-933f-076c2a59dc3a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:55.397638 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-bde667bf-86c4-4131-933f-076c2a59dc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:55.411078 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1060/2141] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:33:55.412026 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1082/2142] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:33:55 2022] POST /identity/v3/users => generated 285 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:55.416048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde667bf-86c4-4131-933f-076c2a59dc3a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:55.417896 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-bde667bf-86c4-4131-933f-076c2a59dc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:55.418145 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28072b0-973a-42b8-9681-e233f56b7ddc None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:55.418437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28072b0-973a-42b8-9681-e233f56b7ddc None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:55.418657 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28072b0-973a-42b8-9681-e233f56b7ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:55.433183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde667bf-86c4-4131-933f-076c2a59dc3a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:55.439556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-f28072b0-973a-42b8-9681-e233f56b7ddc None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:55.440862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcfc7e-0a14-410a-8603-8752aeb865fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:55.441061 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f28072b0-973a-42b8-9681-e233f56b7ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:55.441543 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcfc7e-0a14-410a-8603-8752aeb865fd admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:55.441826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcfc7e-0a14-410a-8603-8752aeb865fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:55.442111 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcfc7e-0a14-410a-8603-8752aeb865fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:55.442911 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcfc7e-0a14-410a-8603-8752aeb865fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:55.444496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcfc7e-0a14-410a-8603-8752aeb865fd admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:55.450704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1083/2143] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:55.458473 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1061/2144] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:33:55.458611 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28072b0-973a-42b8-9681-e233f56b7ddc None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:55.459872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcfc7e-0a14-410a-8603-8752aeb865fd admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:55.469621 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28072b0-973a-42b8-9681-e233f56b7ddc None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:55.475872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9af57e-a1ea-497f-95cd-88346cb5e1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:55.476045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcfc7e-0a14-410a-8603-8752aeb865fd admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:55.476561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9af57e-a1ea-497f-95cd-88346cb5e1c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:55.476847 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9af57e-a1ea-497f-95cd-88346cb5e1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:55.477127 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9af57e-a1ea-497f-95cd-88346cb5e1c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:55.478075 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9af57e-a1ea-497f-95cd-88346cb5e1c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:33:55.478456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9af57e-a1ea-497f-95cd-88346cb5e1c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:55.481293 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9af57e-a1ea-497f-95cd-88346cb5e1c8 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:55.481901 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad127ed4-754b-4fc0-9808-5d74562868fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:55.482727 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad127ed4-754b-4fc0-9808-5d74562868fc admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:55.485767 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad127ed4-754b-4fc0-9808-5d74562868fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:55.485767 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad127ed4-754b-4fc0-9808-5d74562868fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:55.485767 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad127ed4-754b-4fc0-9808-5d74562868fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:55.485767 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad127ed4-754b-4fc0-9808-5d74562868fc admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:55.510667 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1084/2145] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:33:55.514475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1062/2146] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:33:55.514989 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9af57e-a1ea-497f-95cd-88346cb5e1c8 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:55.518687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad127ed4-754b-4fc0-9808-5d74562868fc admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:55.525219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9af57e-a1ea-497f-95cd-88346cb5e1c8 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:55.531833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81216d36-811c-4c68-8bbb-5ecf70460cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:55.532050 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad127ed4-754b-4fc0-9808-5d74562868fc admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:55.532612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81216d36-811c-4c68-8bbb-5ecf70460cbc admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:55.533007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81216d36-811c-4c68-8bbb-5ecf70460cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:55.533430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81216d36-811c-4c68-8bbb-5ecf70460cbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:55.534129 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81216d36-811c-4c68-8bbb-5ecf70460cbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:55.536262 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81216d36-811c-4c68-8bbb-5ecf70460cbc admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:55.543217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f6c26-b342-42de-bc08-b3684672411b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:55.544489 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f6c26-b342-42de-bc08-b3684672411b admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:55.544962 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f6c26-b342-42de-bc08-b3684672411b admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:55.545442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f6c26-b342-42de-bc08-b3684672411b admin admin] PATH_INFO: `/v3/projects/9adb6a1378e04df5978660014d2c44f7/users/ed454faac2814963a6e89c848bdb2dbd/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:55.568731 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f6c26-b342-42de-bc08-b3684672411b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9adb6a1378e04df5978660014d2c44f7, user_id=ed454faac2814963a6e89c848bdb2dbd, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:55.571587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f6c26-b342-42de-bc08-b3684672411b admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:55.704041 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1085/2147] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:33:55 2022] POST /identity/v3/projects => generated 371 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:55.710360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81216d36-811c-4c68-8bbb-5ecf70460cbc admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:55.759275 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1063/2148] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:55 2022] PUT /identity/v3/projects/9adb6a1378e04df5978660014d2c44f7/users/ed454faac2814963a6e89c848bdb2dbd/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:33:55.787359 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cb55b-fdf6-4d51-8cf2-31fa25f6fef9 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:55.787825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cb55b-fdf6-4d51-8cf2-31fa25f6fef9 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:55.788279 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3cb55b-fdf6-4d51-8cf2-31fa25f6fef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:55.792315 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81216d36-811c-4c68-8bbb-5ecf70460cbc admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:55.829369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-bc3cb55b-fdf6-4d51-8cf2-31fa25f6fef9 None None] MFA Rules not processed for user `ed454faac2814963a6e89c848bdb2dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:55.834410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-bc3cb55b-fdf6-4d51-8cf2-31fa25f6fef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:55.858046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131b154-817e-4b73-b432-ccfa9ec78d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:55.858833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131b154-817e-4b73-b432-ccfa9ec78d74 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:55.859152 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131b154-817e-4b73-b432-ccfa9ec78d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:55.860064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131b154-817e-4b73-b432-ccfa9ec78d74 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:55.860993 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131b154-817e-4b73-b432-ccfa9ec78d74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:55.863357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131b154-817e-4b73-b432-ccfa9ec78d74 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:55.909394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1064/2149] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:33:55.925725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cb55b-fdf6-4d51-8cf2-31fa25f6fef9 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:55.927190 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-bc3cb55b-fdf6-4d51-8cf2-31fa25f6fef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:55.992657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3cb55b-fdf6-4d51-8cf2-31fa25f6fef9 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:55.996441 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940a0f4-008c-4647-8b38-7632bc9eedba service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6c13596c2df438294bdac7c52ae8cb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:55.997089 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940a0f4-008c-4647-8b38-7632bc9eedba service glance] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:55.997373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940a0f4-008c-4647-8b38-7632bc9eedba service glance] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:55.998203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940a0f4-008c-4647-8b38-7632bc9eedba service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:55.999970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1086/2150] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:33:55 2022] POST /identity/v3/users => generated 286 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:33:56.003786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131b154-817e-4b73-b432-ccfa9ec78d74 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:56.007437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940a0f4-008c-4647-8b38-7632bc9eedba service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:56.009036 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940a0f4-008c-4647-8b38-7632bc9eedba service glance] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:56.014145 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131b154-817e-4b73-b432-ccfa9ec78d74 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:56.018234 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1065/2151] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:55 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:33:56.020417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2eb7e9-99e8-4ce1-a2ff-8a582ee3af21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:56.021246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eb7e9-99e8-4ce1-a2ff-8a582ee3af21 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:56.021541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eb7e9-99e8-4ce1-a2ff-8a582ee3af21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:56.021824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eb7e9-99e8-4ce1-a2ff-8a582ee3af21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:56.022648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2eb7e9-99e8-4ce1-a2ff-8a582ee3af21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:56.025724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2eb7e9-99e8-4ce1-a2ff-8a582ee3af21 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:56.036461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1087/2152] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:56.040763 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940a0f4-008c-4647-8b38-7632bc9eedba service glance] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:56.054449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940a0f4-008c-4647-8b38-7632bc9eedba service glance] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:56.064583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a34b4f-d760-49c2-b0d3-7094018908d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:56.065703 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a34b4f-d760-49c2-b0d3-7094018908d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:56.075002 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a34b4f-d760-49c2-b0d3-7094018908d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:56.075002 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a34b4f-d760-49c2-b0d3-7094018908d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:56.075002 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a34b4f-d760-49c2-b0d3-7094018908d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:56.075002 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a34b4f-d760-49c2-b0d3-7094018908d4 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:56.087046 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1066/2153] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:33:56.088612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2eb7e9-99e8-4ce1-a2ff-8a582ee3af21 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:56.103074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2eb7e9-99e8-4ce1-a2ff-8a582ee3af21 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:56.109005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c273d8-9936-49aa-afc4-76c2f61b785d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:56.110389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c273d8-9936-49aa-afc4-76c2f61b785d admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:56.110736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c273d8-9936-49aa-afc4-76c2f61b785d admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:56.111055 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c273d8-9936-49aa-afc4-76c2f61b785d admin admin] PATH_INFO: `/v3/projects/1cd8d3e7499342d6820afd0c022033d7/users/bbc3a225e07c4cef9f6b9713fc85942c/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:56.137230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c273d8-9936-49aa-afc4-76c2f61b785d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd8d3e7499342d6820afd0c022033d7, user_id=bbc3a225e07c4cef9f6b9713fc85942c, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:56.142386 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c273d8-9936-49aa-afc4-76c2f61b785d admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:56.173472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:56.224660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1088/2154] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:56 2022] PUT /identity/v3/projects/1cd8d3e7499342d6820afd0c022033d7/users/bbc3a225e07c4cef9f6b9713fc85942c/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:56.226689 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c0660-d202-4a12-ae7b-f012683d7500 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:56.227078 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c0660-d202-4a12-ae7b-f012683d7500 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:56.227888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c0660-d202-4a12-ae7b-f012683d7500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:56.249142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-1a6c0660-d202-4a12-ae7b-f012683d7500 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:56.250778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-1a6c0660-d202-4a12-ae7b-f012683d7500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:56.360719 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1067/2155] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:56.368222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c273d8-9936-49aa-afc4-76c2f61b785d admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:56.370940 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-04c273d8-9936-49aa-afc4-76c2f61b785d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:56.451765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c273d8-9936-49aa-afc4-76c2f61b785d admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:56.455431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cdf78-7875-4d25-ae39-875549f73b86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:56.456077 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cdf78-7875-4d25-ae39-875549f73b86 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:56.456357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cdf78-7875-4d25-ae39-875549f73b86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:56.456676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cdf78-7875-4d25-ae39-875549f73b86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:56.467301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cdf78-7875-4d25-ae39-875549f73b86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:56.468751 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cdf78-7875-4d25-ae39-875549f73b86 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:56.478861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1089/2156] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:33:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:33:56.932997 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:57.467021 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:58.044981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:58.046905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d0c03-fa56-4534-931b-b3d9ce493bcd None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:58.047183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d0c03-fa56-4534-931b-b3d9ce493bcd None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:58.047482 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d0c03-fa56-4534-931b-b3d9ce493bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:58.074529 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-bb0d0c03-fa56-4534-931b-b3d9ce493bcd None None] MFA Rules not processed for user `bbc3a225e07c4cef9f6b9713fc85942c`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:33:58.076447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-bb0d0c03-fa56-4534-931b-b3d9ce493bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:33:58.178627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1068/2157] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:33:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:33:58.183266 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cdf78-7875-4d25-ae39-875549f73b86 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:58.193738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cdf78-7875-4d25-ae39-875549f73b86 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:58.198611 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a322ffc-281c-4b08-958a-986a81348405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:58.199265 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a322ffc-281c-4b08-958a-986a81348405 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:58.200262 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a322ffc-281c-4b08-958a-986a81348405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:58.200262 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a322ffc-281c-4b08-958a-986a81348405 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:58.201200 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a322ffc-281c-4b08-958a-986a81348405 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:58.203296 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a322ffc-281c-4b08-958a-986a81348405 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:58.331439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1090/2158] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:33:58 2022] POST /identity/v3/projects => generated 371 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:58.334591 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d0c03-fa56-4534-931b-b3d9ce493bcd None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:58.345858 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d0c03-fa56-4534-931b-b3d9ce493bcd None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:58.405378 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ce94c-a568-4e60-8d74-0992a6348b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:58.406067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ce94c-a568-4e60-8d74-0992a6348b2a admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:58.406379 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ce94c-a568-4e60-8d74-0992a6348b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:58.406979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ce94c-a568-4e60-8d74-0992a6348b2a admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:58.407357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ce94c-a568-4e60-8d74-0992a6348b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:58.409399 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ce94c-a568-4e60-8d74-0992a6348b2a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:58.489810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:33:58.533072 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1069/2159] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:33:58 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:33:58.536355 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a322ffc-281c-4b08-958a-986a81348405 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:58.549431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a322ffc-281c-4b08-958a-986a81348405 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:58.554812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d1470-4c85-4221-b553-4910965e8e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:58.555748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d1470-4c85-4221-b553-4910965e8e0f admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:58.555858 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d1470-4c85-4221-b553-4910965e8e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:58.556355 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d1470-4c85-4221-b553-4910965e8e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:58.556993 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d1470-4c85-4221-b553-4910965e8e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:58.558718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d1470-4c85-4221-b553-4910965e8e0f admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:58.571371 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1091/2160] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:58.578010 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ce94c-a568-4e60-8d74-0992a6348b2a admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:58.591022 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ce94c-a568-4e60-8d74-0992a6348b2a admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:58.597628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289fbad-44f0-4c45-9fc9-beaaf3071cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:58.603140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289fbad-44f0-4c45-9fc9-beaaf3071cde admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:58.603140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289fbad-44f0-4c45-9fc9-beaaf3071cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:58.603140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289fbad-44f0-4c45-9fc9-beaaf3071cde admin admin] PATH_INFO: `/v3/projects/18b8bc855d7246b2a773de85f151ec63/users/389c21cefe094be18841369f0f1b8570/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:58.616370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289fbad-44f0-4c45-9fc9-beaaf3071cde admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b8bc855d7246b2a773de85f151ec63, user_id=389c21cefe094be18841369f0f1b8570, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:58.619386 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289fbad-44f0-4c45-9fc9-beaaf3071cde admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:58.718520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1070/2161] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:58 2022] PUT /identity/v3/projects/18b8bc855d7246b2a773de85f151ec63/users/389c21cefe094be18841369f0f1b8570/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:33:58.719488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d1470-4c85-4221-b553-4910965e8e0f admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:58.748933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d1470-4c85-4221-b553-4910965e8e0f admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:58.813830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cacaacb-4634-4d99-abb8-c741df6bdd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:58.814690 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacaacb-4634-4d99-abb8-c741df6bdd00 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:58.815085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacaacb-4634-4d99-abb8-c741df6bdd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:58.815409 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacaacb-4634-4d99-abb8-c741df6bdd00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:58.816165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cacaacb-4634-4d99-abb8-c741df6bdd00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:58.817832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cacaacb-4634-4d99-abb8-c741df6bdd00 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:58.829062 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1092/2162] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:33:58.841922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289fbad-44f0-4c45-9fc9-beaaf3071cde admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:33:58.860310 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289fbad-44f0-4c45-9fc9-beaaf3071cde admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:33:58.865648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f66c43-9309-4658-bfb5-940bc33b5e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:33:58.866578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f66c43-9309-4658-bfb5-940bc33b5e29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:33:58.869986 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f66c43-9309-4658-bfb5-940bc33b5e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:33:58.869986 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f66c43-9309-4658-bfb5-940bc33b5e29 admin admin] PATH_INFO: `/v3/projects/18b8bc855d7246b2a773de85f151ec63/users/389c21cefe094be18841369f0f1b8570/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:33:58.869986 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f66c43-9309-4658-bfb5-940bc33b5e29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b8bc855d7246b2a773de85f151ec63, user_id=389c21cefe094be18841369f0f1b8570, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:33:58.872262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f66c43-9309-4658-bfb5-940bc33b5e29 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:33:58.885218 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1071/2163] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:33:58 2022] PUT /identity/v3/projects/18b8bc855d7246b2a773de85f151ec63/users/389c21cefe094be18841369f0f1b8570/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:00.485347 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc05a6c-130a-4a1c-a3ef-3a9dceabb4f2 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:00.486149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc05a6c-130a-4a1c-a3ef-3a9dceabb4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:00.486149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc05a6c-130a-4a1c-a3ef-3a9dceabb4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:00.553412 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-fdc05a6c-130a-4a1c-a3ef-3a9dceabb4f2 None None] MFA Rules not processed for user `389c21cefe094be18841369f0f1b8570`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:00.553884 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-fdc05a6c-130a-4a1c-a3ef-3a9dceabb4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:00.769716 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1093/2164] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:00 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:01.166915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f66c43-9309-4658-bfb5-940bc33b5e29 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:01.168744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-69f66c43-9309-4658-bfb5-940bc33b5e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:01.248098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f66c43-9309-4658-bfb5-940bc33b5e29 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:01.252076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93868c7-4191-4dbf-9fb0-d9e66da51d73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:01.252737 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93868c7-4191-4dbf-9fb0-d9e66da51d73 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:01.253019 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93868c7-4191-4dbf-9fb0-d9e66da51d73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:01.253942 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93868c7-4191-4dbf-9fb0-d9e66da51d73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:01.324451 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93868c7-4191-4dbf-9fb0-d9e66da51d73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:01.326588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93868c7-4191-4dbf-9fb0-d9e66da51d73 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:01.336066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1072/2165] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:06.582141 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc05a6c-130a-4a1c-a3ef-3a9dceabb4f2 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:06.603046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc05a6c-130a-4a1c-a3ef-3a9dceabb4f2 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:06.603046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1828ba0-ed6f-40cd-8553-98c80b902464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:06.603046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1828ba0-ed6f-40cd-8553-98c80b902464 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:06.603046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1828ba0-ed6f-40cd-8553-98c80b902464 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:06.603361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1828ba0-ed6f-40cd-8553-98c80b902464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:06.611719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1828ba0-ed6f-40cd-8553-98c80b902464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:06.613173 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1828ba0-ed6f-40cd-8553-98c80b902464 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:06.623800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1094/2166] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:06 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:34:09.852429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93868c7-4191-4dbf-9fb0-d9e66da51d73 service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:09.854014 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e93868c7-4191-4dbf-9fb0-d9e66da51d73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:09.900462 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:09.914824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c9102-5284-447a-aa79-02fb7a532f49 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:09.915746 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c9102-5284-447a-aa79-02fb7a532f49 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:09.916202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c9102-5284-447a-aa79-02fb7a532f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:09.938674 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-fe3c9102-5284-447a-aa79-02fb7a532f49 None None] MFA Rules not processed for user `ebbcfe6341a3486d93e12c09f83954dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:09.939927 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-fe3c9102-5284-447a-aa79-02fb7a532f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:09.946188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93868c7-4191-4dbf-9fb0-d9e66da51d73 service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:09.951486 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8cdfc-e385-4d8e-86e9-b0d52c958917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:09.952616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8cdfc-e385-4d8e-86e9-b0d52c958917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:09.953082 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8cdfc-e385-4d8e-86e9-b0d52c958917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:09.953452 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8cdfc-e385-4d8e-86e9-b0d52c958917 admin admin] PATH_INFO: `/v3/users/d5c945c9e9874c14b53c212c1d836e11` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:09.956464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8cdfc-e385-4d8e-86e9-b0d52c958917 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5c945c9e9874c14b53c212c1d836e11)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:09.958150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8cdfc-e385-4d8e-86e9-b0d52c958917 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:09.958765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:10.013044 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1095/2167] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:34:10.143913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-fee8cdfc-e385-4d8e-86e9-b0d52c958917 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5c945c9e9874c14b53c212c1d836e11'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:10.301130 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-fee8cdfc-e385-4d8e-86e9-b0d52c958917 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5c945c9e9874c14b53c212c1d836e11'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:10.301828 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-fee8cdfc-e385-4d8e-86e9-b0d52c958917 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5c945c9e9874c14b53c212c1d836e11'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:10.314780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-fee8cdfc-e385-4d8e-86e9-b0d52c958917 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5c945c9e9874c14b53c212c1d836e11'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:10.322597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-fee8cdfc-e385-4d8e-86e9-b0d52c958917 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5c945c9e9874c14b53c212c1d836e11'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:10.329273 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1073/2168] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:34:09 2022] DELETE /identity/v3/users/d5c945c9e9874c14b53c212c1d836e11 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:10.334408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ed476-e773-48b0-872f-e24570732ad7 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:10.335078 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ed476-e773-48b0-872f-e24570732ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:10.335512 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ed476-e773-48b0-872f-e24570732ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:10.370508 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-eb7ed476-e773-48b0-872f-e24570732ad7 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:10.370508 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-eb7ed476-e773-48b0-872f-e24570732ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:10.449456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1096/2169] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:10.460939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8cdfc-e385-4d8e-86e9-b0d52c958917 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:10.467523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-fee8cdfc-e385-4d8e-86e9-b0d52c958917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:10.558953 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8cdfc-e385-4d8e-86e9-b0d52c958917 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:10.564008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678e327-8631-4587-a0ad-c92e1b3ea00f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:10.564158 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678e327-8631-4587-a0ad-c92e1b3ea00f service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:10.564448 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678e327-8631-4587-a0ad-c92e1b3ea00f service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:10.564811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678e327-8631-4587-a0ad-c92e1b3ea00f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:10.584800 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678e327-8631-4587-a0ad-c92e1b3ea00f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:10.584800 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678e327-8631-4587-a0ad-c92e1b3ea00f service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:10.596705 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1074/2170] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:10.866625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ed476-e773-48b0-872f-e24570732ad7 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:10.868088 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-eb7ed476-e773-48b0-872f-e24570732ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:10.883834 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ed476-e773-48b0-872f-e24570732ad7 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:10.916356 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f727d-0c38-49c7-87eb-c6363969b7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:10.917184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f727d-0c38-49c7-87eb-c6363969b7b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:10.917979 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f727d-0c38-49c7-87eb-c6363969b7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:10.919668 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f727d-0c38-49c7-87eb-c6363969b7b2 admin admin] PATH_INFO: `/v3/users/192b8a31ff70484b9d0fc59c9c4ec075` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:10.924469 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365f727d-0c38-49c7-87eb-c6363969b7b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=192b8a31ff70484b9d0fc59c9c4ec075)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:10.933223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365f727d-0c38-49c7-87eb-c6363969b7b2 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:11.180384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-365f727d-0c38-49c7-87eb-c6363969b7b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '192b8a31ff70484b9d0fc59c9c4ec075'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:11.300540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-365f727d-0c38-49c7-87eb-c6363969b7b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '192b8a31ff70484b9d0fc59c9c4ec075'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:11.311093 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678e327-8631-4587-a0ad-c92e1b3ea00f service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:11.313620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f678e327-8631-4587-a0ad-c92e1b3ea00f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:11.336666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-365f727d-0c38-49c7-87eb-c6363969b7b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '192b8a31ff70484b9d0fc59c9c4ec075'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:11.338362 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-365f727d-0c38-49c7-87eb-c6363969b7b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '192b8a31ff70484b9d0fc59c9c4ec075'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:11.359570 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678e327-8631-4587-a0ad-c92e1b3ea00f service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:11.378583 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-365f727d-0c38-49c7-87eb-c6363969b7b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '192b8a31ff70484b9d0fc59c9c4ec075'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:11.378779 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73f800-9631-471a-bcce-3f38964dc6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:11.379673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73f800-9631-471a-bcce-3f38964dc6f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:11.379961 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73f800-9631-471a-bcce-3f38964dc6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:11.380244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73f800-9631-471a-bcce-3f38964dc6f8 admin admin] PATH_INFO: `/v3/projects/c2759635a13e44ed9518b88baa879471` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:11.383011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73f800-9631-471a-bcce-3f38964dc6f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2759635a13e44ed9518b88baa879471)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:11.384511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73f800-9631-471a-bcce-3f38964dc6f8 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:11.391864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1097/2171] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:34:10 2022] DELETE /identity/v3/users/192b8a31ff70484b9d0fc59c9c4ec075 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:11.395653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392a5d0-4a1e-4474-bc70-e7c05ae758d9 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:11.396061 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392a5d0-4a1e-4474-bc70-e7c05ae758d9 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:11.397399 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392a5d0-4a1e-4474-bc70-e7c05ae758d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:11.430396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6392a5d0-4a1e-4474-bc70-e7c05ae758d9 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:11.431075 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6392a5d0-4a1e-4474-bc70-e7c05ae758d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:11.470840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:11.499998 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1098/2172] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:11.512931 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392a5d0-4a1e-4474-bc70-e7c05ae758d9 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:11.514723 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6392a5d0-4a1e-4474-bc70-e7c05ae758d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:11.564612 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-3f73f800-9631-471a-bcce-3f38964dc6f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2759635a13e44ed9518b88baa879471', 'request_id': 'req-3f73f800-9631-471a-bcce-3f38964dc6f8'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:11.570127 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:11.585484 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392a5d0-4a1e-4474-bc70-e7c05ae758d9 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:11.639223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-3f73f800-9631-471a-bcce-3f38964dc6f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2759635a13e44ed9518b88baa879471', 'request_id': 'req-3f73f800-9631-471a-bcce-3f38964dc6f8'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:11.640996 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897189b4-f6ff-4a4c-91a5-45d55147eda6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:11.642589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897189b4-f6ff-4a4c-91a5-45d55147eda6 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:11.652102 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897189b4-f6ff-4a4c-91a5-45d55147eda6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:11.652102 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897189b4-f6ff-4a4c-91a5-45d55147eda6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:11.652320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-3f73f800-9631-471a-bcce-3f38964dc6f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2759635a13e44ed9518b88baa879471', 'request_id': 'req-3f73f800-9631-471a-bcce-3f38964dc6f8'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:11.658149 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-3f73f800-9631-471a-bcce-3f38964dc6f8 admin admin] The token cache is being invalidate because project c2759635a13e44ed9518b88baa879471 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:34:11.658489 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-3f73f800-9631-471a-bcce-3f38964dc6f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:11.659241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-3f73f800-9631-471a-bcce-3f38964dc6f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:11.660790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1075/2173] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:34:11 2022] DELETE /identity/v3/projects/c2759635a13e44ed9518b88baa879471 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:11.665790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba3f89-fff7-4449-b010-3fb7ad53fd86 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:11.666080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba3f89-fff7-4449-b010-3fb7ad53fd86 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:11.666391 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba3f89-fff7-4449-b010-3fb7ad53fd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:11.684725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-9cba3f89-fff7-4449-b010-3fb7ad53fd86 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:11.685962 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-9cba3f89-fff7-4449-b010-3fb7ad53fd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:11.717033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897189b4-f6ff-4a4c-91a5-45d55147eda6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:11.718592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897189b4-f6ff-4a4c-91a5-45d55147eda6 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:11.729357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1099/2174] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:34:11.770858 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1076/2175] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:34:11.780059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc176d9b-9cb6-4ea0-b866-4f227fa77c86 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:11.780059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc176d9b-9cb6-4ea0-b866-4f227fa77c86 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:11.780460 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc176d9b-9cb6-4ea0-b866-4f227fa77c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:11.800936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-fc176d9b-9cb6-4ea0-b866-4f227fa77c86 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:11.802213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-fc176d9b-9cb6-4ea0-b866-4f227fa77c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:11.811421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1100/2176] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:11.815456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba3f89-fff7-4449-b010-3fb7ad53fd86 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:11.827103 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba3f89-fff7-4449-b010-3fb7ad53fd86 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:11.832600 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d1bd4-1ab0-4182-a268-fd597e635c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:11.833324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d1bd4-1ab0-4182-a268-fd597e635c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:11.833621 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d1bd4-1ab0-4182-a268-fd597e635c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:11.833901 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d1bd4-1ab0-4182-a268-fd597e635c3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:11.834897 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d1bd4-1ab0-4182-a268-fd597e635c3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:34:11.844345 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d1bd4-1ab0-4182-a268-fd597e635c3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:11.844345 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d1bd4-1ab0-4182-a268-fd597e635c3a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:11.849815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1077/2177] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:34:11.854165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc176d9b-9cb6-4ea0-b866-4f227fa77c86 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:11.864113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc176d9b-9cb6-4ea0-b866-4f227fa77c86 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:11.870588 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf47db-d00e-4940-a411-03c94d2eb05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:11.871346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf47db-d00e-4940-a411-03c94d2eb05f admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:11.871633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf47db-d00e-4940-a411-03c94d2eb05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:11.875838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf47db-d00e-4940-a411-03c94d2eb05f admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:11.877405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf47db-d00e-4940-a411-03c94d2eb05f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:11.879728 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf47db-d00e-4940-a411-03c94d2eb05f admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:12.043014 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1101/2178] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:34:11 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:12.047077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d1bd4-1ab0-4182-a268-fd597e635c3a admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:12.062226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d1bd4-1ab0-4182-a268-fd597e635c3a admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:12.164744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1ad82-461b-4f10-a6a7-fec1babe4b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:12.165427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1ad82-461b-4f10-a6a7-fec1babe4b20 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:12.165715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1ad82-461b-4f10-a6a7-fec1babe4b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:12.166102 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1ad82-461b-4f10-a6a7-fec1babe4b20 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:12.166867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1ad82-461b-4f10-a6a7-fec1babe4b20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:12.172744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1ad82-461b-4f10-a6a7-fec1babe4b20 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:12.291998 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf47db-d00e-4940-a411-03c94d2eb05f admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:12.294548 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2abf47db-d00e-4940-a411-03c94d2eb05f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:12.315700 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1078/2179] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:34:12 2022] POST /identity/v3/users => generated 292 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:12.319615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1ad82-461b-4f10-a6a7-fec1babe4b20 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:12.320585 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf47db-d00e-4940-a411-03c94d2eb05f admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:12.327655 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:12.330036 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:12.330699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:12.331407 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274 admin admin] PATH_INFO: `/v3/projects/94ddbdb125164a1fbbc85e5295c3ee93` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:12.342508 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94ddbdb125164a1fbbc85e5295c3ee93)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:12.343808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:12.345684 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1ad82-461b-4f10-a6a7-fec1babe4b20 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:12.352184 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df3b72-a096-4b87-8bf7-2d1457e56951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:12.353166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df3b72-a096-4b87-8bf7-2d1457e56951 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:12.353628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df3b72-a096-4b87-8bf7-2d1457e56951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:12.366015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df3b72-a096-4b87-8bf7-2d1457e56951 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:12.366913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df3b72-a096-4b87-8bf7-2d1457e56951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:12.374572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df3b72-a096-4b87-8bf7-2d1457e56951 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:12.393978 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1079/2180] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:12.404337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df3b72-a096-4b87-8bf7-2d1457e56951 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:12.420265 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df3b72-a096-4b87-8bf7-2d1457e56951 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:12.427516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03f709-5066-4f9d-a784-0017dd142bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:12.428493 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03f709-5066-4f9d-a784-0017dd142bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:12.429050 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03f709-5066-4f9d-a784-0017dd142bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:12.429504 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03f709-5066-4f9d-a784-0017dd142bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:12.434277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03f709-5066-4f9d-a784-0017dd142bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:12.434277 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03f709-5066-4f9d-a784-0017dd142bb7 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:12.446949 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1080/2181] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:12.451710 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03f709-5066-4f9d-a784-0017dd142bb7 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:12.483445 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03f709-5066-4f9d-a784-0017dd142bb7 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:12.488766 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94ddbdb125164a1fbbc85e5295c3ee93', 'request_id': 'req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:12.556192 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fc924-39e7-42eb-b50a-e41e0e253d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:12.560808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fc924-39e7-42eb-b50a-e41e0e253d50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:12.560808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fc924-39e7-42eb-b50a-e41e0e253d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:12.560808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fc924-39e7-42eb-b50a-e41e0e253d50 admin admin] PATH_INFO: `/v3/projects/8793a880033e400d89ac8de5a30003b7/users/dce92cce74c84abcbec76358c860535f/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:12.583737 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503fc924-39e7-42eb-b50a-e41e0e253d50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8793a880033e400d89ac8de5a30003b7, user_id=dce92cce74c84abcbec76358c860535f, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:12.586529 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503fc924-39e7-42eb-b50a-e41e0e253d50 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:12.591612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94ddbdb125164a1fbbc85e5295c3ee93', 'request_id': 'req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:12.599840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94ddbdb125164a1fbbc85e5295c3ee93', 'request_id': 'req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:12.614288 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274 admin admin] The token cache is being invalidate because project 94ddbdb125164a1fbbc85e5295c3ee93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:34:12.614594 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:12.618304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:12.618304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1102/2182] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:34:12 2022] DELETE /identity/v3/projects/94ddbdb125164a1fbbc85e5295c3ee93 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:34:12.620887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:12.623037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:12.635933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f67e5-2a7d-4ff7-8029-1edf9c68c274 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:12.640720 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299bbd9-e21a-4c71-8f7f-578c1ef49c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:12.641612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299bbd9-e21a-4c71-8f7f-578c1ef49c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:12.641898 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299bbd9-e21a-4c71-8f7f-578c1ef49c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:12.642181 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299bbd9-e21a-4c71-8f7f-578c1ef49c33 admin admin] PATH_INFO: `/v3/users/49b85d860db44e2aad1173dde389bfe8` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:12.643461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299bbd9-e21a-4c71-8f7f-578c1ef49c33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49b85d860db44e2aad1173dde389bfe8)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:12.645408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299bbd9-e21a-4c71-8f7f-578c1ef49c33 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:12.689237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1081/2183] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:12 2022] PUT /identity/v3/projects/8793a880033e400d89ac8de5a30003b7/users/dce92cce74c84abcbec76358c860535f/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:34:12.693263 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e483a6a-1586-402d-81d6-83b8401ac649 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:12.693556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e483a6a-1586-402d-81d6-83b8401ac649 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:12.693836 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e483a6a-1586-402d-81d6-83b8401ac649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:12.711902 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-6e483a6a-1586-402d-81d6-83b8401ac649 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:12.713224 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-6e483a6a-1586-402d-81d6-83b8401ac649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:12.772522 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1082/2184] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:34:12.783026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e483a6a-1586-402d-81d6-83b8401ac649 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:12.784661 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-6e483a6a-1586-402d-81d6-83b8401ac649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:12.846772 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e483a6a-1586-402d-81d6-83b8401ac649 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:12.850602 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462497a-cd54-499b-8bfc-1fffd8afe985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:12.851254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462497a-cd54-499b-8bfc-1fffd8afe985 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:12.851541 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462497a-cd54-499b-8bfc-1fffd8afe985 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:12.851844 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462497a-cd54-499b-8bfc-1fffd8afe985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:12.862055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4462497a-cd54-499b-8bfc-1fffd8afe985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:12.863516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4462497a-cd54-499b-8bfc-1fffd8afe985 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:12.877167 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1083/2185] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:34:12.895426 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:12.952143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-a299bbd9-e21a-4c71-8f7f-578c1ef49c33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49b85d860db44e2aad1173dde389bfe8'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:13.039137 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:13.040128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-a299bbd9-e21a-4c71-8f7f-578c1ef49c33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49b85d860db44e2aad1173dde389bfe8'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:13.057901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-a299bbd9-e21a-4c71-8f7f-578c1ef49c33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49b85d860db44e2aad1173dde389bfe8'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:13.059077 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-a299bbd9-e21a-4c71-8f7f-578c1ef49c33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49b85d860db44e2aad1173dde389bfe8'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:13.085346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-a299bbd9-e21a-4c71-8f7f-578c1ef49c33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49b85d860db44e2aad1173dde389bfe8'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:13.095191 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1103/2186] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:34:12 2022] DELETE /identity/v3/users/49b85d860db44e2aad1173dde389bfe8 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:34:13.628921 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462497a-cd54-499b-8bfc-1fffd8afe985 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:13.639687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462497a-cd54-499b-8bfc-1fffd8afe985 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:13.717256 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fab328-fb29-4f71-ad16-c5e3b8efa6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:13.718054 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fab328-fb29-4f71-ad16-c5e3b8efa6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:13.718370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fab328-fb29-4f71-ad16-c5e3b8efa6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:13.718643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fab328-fb29-4f71-ad16-c5e3b8efa6b9 admin admin] PATH_INFO: `/v3/projects/116f245900cf4d22b211ae8c22e23882` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:13.719862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fab328-fb29-4f71-ad16-c5e3b8efa6b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=116f245900cf4d22b211ae8c22e23882)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:13.721782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fab328-fb29-4f71-ad16-c5e3b8efa6b9 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:13.890685 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-19fab328-fb29-4f71-ad16-c5e3b8efa6b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '116f245900cf4d22b211ae8c22e23882', 'request_id': 'req-19fab328-fb29-4f71-ad16-c5e3b8efa6b9'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:14.015057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-19fab328-fb29-4f71-ad16-c5e3b8efa6b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '116f245900cf4d22b211ae8c22e23882', 'request_id': 'req-19fab328-fb29-4f71-ad16-c5e3b8efa6b9'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:14.022852 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-19fab328-fb29-4f71-ad16-c5e3b8efa6b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '116f245900cf4d22b211ae8c22e23882', 'request_id': 'req-19fab328-fb29-4f71-ad16-c5e3b8efa6b9'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:14.033546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-19fab328-fb29-4f71-ad16-c5e3b8efa6b9 admin admin] The token cache is being invalidate because project 116f245900cf4d22b211ae8c22e23882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:34:14.033838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-19fab328-fb29-4f71-ad16-c5e3b8efa6b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:14.034526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-19fab328-fb29-4f71-ad16-c5e3b8efa6b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:14.036177 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1084/2187] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:34:13 2022] DELETE /identity/v3/projects/116f245900cf4d22b211ae8c22e23882 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:14.041361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8fe5c-a8e1-4fb5-a46a-377f84390a44 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:14.042032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8fe5c-a8e1-4fb5-a46a-377f84390a44 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:14.042361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8fe5c-a8e1-4fb5-a46a-377f84390a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:14.062334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-bef8fe5c-a8e1-4fb5-a46a-377f84390a44 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:14.063619 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-bef8fe5c-a8e1-4fb5-a46a-377f84390a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:14.133382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1104/2188] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:14.141610 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313103f-f9a0-4944-8588-c7960bb4be53 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:14.141897 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313103f-f9a0-4944-8588-c7960bb4be53 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:14.142176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313103f-f9a0-4944-8588-c7960bb4be53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:14.171192 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-5313103f-f9a0-4944-8588-c7960bb4be53 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:14.172838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5313103f-f9a0-4944-8588-c7960bb4be53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:14.189590 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1085/2189] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:34:14.193970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8fe5c-a8e1-4fb5-a46a-377f84390a44 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:14.203802 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8fe5c-a8e1-4fb5-a46a-377f84390a44 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:14.208463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb67e8-1e4c-4b5c-889b-fc8da4836e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:14.209123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb67e8-1e4c-4b5c-889b-fc8da4836e40 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:14.209413 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb67e8-1e4c-4b5c-889b-fc8da4836e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:14.209717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb67e8-1e4c-4b5c-889b-fc8da4836e40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:14.210479 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb67e8-1e4c-4b5c-889b-fc8da4836e40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:34:14.210794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb67e8-1e4c-4b5c-889b-fc8da4836e40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:14.212400 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb67e8-1e4c-4b5c-889b-fc8da4836e40 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:14.221734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:14.225140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1105/2190] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:34:14.229440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313103f-f9a0-4944-8588-c7960bb4be53 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:14.239366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313103f-f9a0-4944-8588-c7960bb4be53 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:14.248259 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6cacda-ee4c-45d5-969c-f955def4c010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:14.248259 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cacda-ee4c-45d5-969c-f955def4c010 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:14.248259 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cacda-ee4c-45d5-969c-f955def4c010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:14.248259 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cacda-ee4c-45d5-969c-f955def4c010 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:14.248705 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6cacda-ee4c-45d5-969c-f955def4c010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:14.248705 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6cacda-ee4c-45d5-969c-f955def4c010 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:14.306157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:14.342636 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1086/2191] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:34:14 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:34:14.346902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb67e8-1e4c-4b5c-889b-fc8da4836e40 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:14.357541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb67e8-1e4c-4b5c-889b-fc8da4836e40 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:14.442791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623e3ad-6407-426a-baf0-c73a3e013817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:14.443498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623e3ad-6407-426a-baf0-c73a3e013817 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:14.443804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623e3ad-6407-426a-baf0-c73a3e013817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:14.444074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623e3ad-6407-426a-baf0-c73a3e013817 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:14.444796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623e3ad-6407-426a-baf0-c73a3e013817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:14.450686 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623e3ad-6407-426a-baf0-c73a3e013817 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:14.561935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1106/2192] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:34:14 2022] POST /identity/v3/users => generated 294 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:14.563228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6cacda-ee4c-45d5-969c-f955def4c010 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:14.576838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6cacda-ee4c-45d5-969c-f955def4c010 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:14.582819 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2e3a8-33f9-4f38-b2a8-d5bb694b2ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:14.583487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2e3a8-33f9-4f38-b2a8-d5bb694b2ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:14.583773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2e3a8-33f9-4f38-b2a8-d5bb694b2ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:14.584048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2e3a8-33f9-4f38-b2a8-d5bb694b2ce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:14.585915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c2e3a8-33f9-4f38-b2a8-d5bb694b2ce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:14.588647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c2e3a8-33f9-4f38-b2a8-d5bb694b2ce7 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:14.609856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1087/2193] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:14.614001 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623e3ad-6407-426a-baf0-c73a3e013817 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:14.624799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623e3ad-6407-426a-baf0-c73a3e013817 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:14.630046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1a883-3f77-44b3-b139-548d13cb4110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:14.634968 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1a883-3f77-44b3-b139-548d13cb4110 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:14.634968 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1a883-3f77-44b3-b139-548d13cb4110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:14.634968 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1a883-3f77-44b3-b139-548d13cb4110 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:14.634968 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1a883-3f77-44b3-b139-548d13cb4110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:14.634968 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1a883-3f77-44b3-b139-548d13cb4110 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:14.646229 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1107/2194] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:14.650666 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2e3a8-33f9-4f38-b2a8-d5bb694b2ce7 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:14.661354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2e3a8-33f9-4f38-b2a8-d5bb694b2ce7 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:14.666332 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86b7fc-b2ea-425a-8c01-e17367c3eca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:14.667222 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86b7fc-b2ea-425a-8c01-e17367c3eca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:14.667507 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86b7fc-b2ea-425a-8c01-e17367c3eca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:14.667910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86b7fc-b2ea-425a-8c01-e17367c3eca3 admin admin] PATH_INFO: `/v3/projects/e6f43de9b423400dab83b96e5b4038c9/users/1c159a2e623d48269a8607de341a98f8/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:14.683355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86b7fc-b2ea-425a-8c01-e17367c3eca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6f43de9b423400dab83b96e5b4038c9, user_id=1c159a2e623d48269a8607de341a98f8, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:14.686023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86b7fc-b2ea-425a-8c01-e17367c3eca3 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:14.807634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1088/2195] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:14 2022] PUT /identity/v3/projects/e6f43de9b423400dab83b96e5b4038c9/users/1c159a2e623d48269a8607de341a98f8/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:14.814913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619daec-aef6-49f6-b5e0-7e9bca0c56ec None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:14.815206 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619daec-aef6-49f6-b5e0-7e9bca0c56ec None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:14.815488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619daec-aef6-49f6-b5e0-7e9bca0c56ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:14.847159 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-1619daec-aef6-49f6-b5e0-7e9bca0c56ec None None] MFA Rules not processed for user `1c159a2e623d48269a8607de341a98f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:14.848584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1619daec-aef6-49f6-b5e0-7e9bca0c56ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:14.927169 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1108/2196] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:14.931556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86b7fc-b2ea-425a-8c01-e17367c3eca3 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:15.006539 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86b7fc-b2ea-425a-8c01-e17367c3eca3 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:15.014459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36040c42-ac93-495e-98c2-ec810bc58860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:15.015660 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36040c42-ac93-495e-98c2-ec810bc58860 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:15.016135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36040c42-ac93-495e-98c2-ec810bc58860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:15.016597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36040c42-ac93-495e-98c2-ec810bc58860 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:15.018178 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36040c42-ac93-495e-98c2-ec810bc58860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:15.020342 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36040c42-ac93-495e-98c2-ec810bc58860 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:15.116994 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1089/2197] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:34:14 2022] POST /identity/v3/projects => generated 387 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:15.120962 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619daec-aef6-49f6-b5e0-7e9bca0c56ec None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:15.133064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619daec-aef6-49f6-b5e0-7e9bca0c56ec None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:15.202338 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6752658-03e5-405c-a42f-082d2cbf1a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:15.203041 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6752658-03e5-405c-a42f-082d2cbf1a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:15.203327 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6752658-03e5-405c-a42f-082d2cbf1a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:15.203607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6752658-03e5-405c-a42f-082d2cbf1a5b admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:15.205629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6752658-03e5-405c-a42f-082d2cbf1a5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:15.206864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6752658-03e5-405c-a42f-082d2cbf1a5b admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:15.291271 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1109/2198] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:34:15 2022] POST /identity/v3/users => generated 300 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:15.295445 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36040c42-ac93-495e-98c2-ec810bc58860 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:15.306465 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36040c42-ac93-495e-98c2-ec810bc58860 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:15.312464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52987c4f-eeae-48b8-8417-78d68dde8f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:15.313269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52987c4f-eeae-48b8-8417-78d68dde8f9a admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:15.313667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52987c4f-eeae-48b8-8417-78d68dde8f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:15.313959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52987c4f-eeae-48b8-8417-78d68dde8f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:15.314759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52987c4f-eeae-48b8-8417-78d68dde8f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:15.316381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52987c4f-eeae-48b8-8417-78d68dde8f9a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:15.331622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1090/2199] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:15.335572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6752658-03e5-405c-a42f-082d2cbf1a5b admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:15.351264 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6752658-03e5-405c-a42f-082d2cbf1a5b admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:15.357382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98498e0e-2c5d-4f7b-b43b-839251a415d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:15.358498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98498e0e-2c5d-4f7b-b43b-839251a415d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:15.359092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98498e0e-2c5d-4f7b-b43b-839251a415d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:15.359092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98498e0e-2c5d-4f7b-b43b-839251a415d4 admin admin] PATH_INFO: `/v3/projects/9fcddac2fa564af2b3e187ef740f6ba1/users/2a102e6144574c378cf94cf1a8aaad51/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:15.377507 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98498e0e-2c5d-4f7b-b43b-839251a415d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fcddac2fa564af2b3e187ef740f6ba1, user_id=2a102e6144574c378cf94cf1a8aaad51, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:15.380850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98498e0e-2c5d-4f7b-b43b-839251a415d4 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:15.392039 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:15.467635 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1110/2200] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:15 2022] PUT /identity/v3/projects/9fcddac2fa564af2b3e187ef740f6ba1/users/2a102e6144574c378cf94cf1a8aaad51/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:15.472160 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52987c4f-eeae-48b8-8417-78d68dde8f9a admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:15.480198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:15.486347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52987c4f-eeae-48b8-8417-78d68dde8f9a admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:15.564275 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3eaa7-b28e-4704-84e0-600759ce75cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:15.564889 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3eaa7-b28e-4704-84e0-600759ce75cc admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:15.565209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3eaa7-b28e-4704-84e0-600759ce75cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:15.565489 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3eaa7-b28e-4704-84e0-600759ce75cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:15.566236 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3eaa7-b28e-4704-84e0-600759ce75cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:15.567879 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3eaa7-b28e-4704-84e0-600759ce75cc admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:15.583506 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1091/2201] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:15.589089 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98498e0e-2c5d-4f7b-b43b-839251a415d4 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:15.602193 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98498e0e-2c5d-4f7b-b43b-839251a415d4 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:15.608653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481bde7-40ba-47e7-995f-253fda417f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:15.609381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481bde7-40ba-47e7-995f-253fda417f01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:15.609669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481bde7-40ba-47e7-995f-253fda417f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:15.610019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481bde7-40ba-47e7-995f-253fda417f01 admin admin] PATH_INFO: `/v3/projects/9fcddac2fa564af2b3e187ef740f6ba1/users/2a102e6144574c378cf94cf1a8aaad51/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:15.612556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481bde7-40ba-47e7-995f-253fda417f01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fcddac2fa564af2b3e187ef740f6ba1, user_id=2a102e6144574c378cf94cf1a8aaad51, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:15.615621 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481bde7-40ba-47e7-995f-253fda417f01 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:15.639354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1111/2202] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:15 2022] PUT /identity/v3/projects/9fcddac2fa564af2b3e187ef740f6ba1/users/2a102e6144574c378cf94cf1a8aaad51/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:15.640951 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa8f77-f403-4ba7-9b29-3f1151b46c53 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:15.641324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa8f77-f403-4ba7-9b29-3f1151b46c53 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:15.641514 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa8f77-f403-4ba7-9b29-3f1151b46c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:15.681075 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-2eaa8f77-f403-4ba7-9b29-3f1151b46c53 None None] MFA Rules not processed for user `2a102e6144574c378cf94cf1a8aaad51`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:15.683459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-2eaa8f77-f403-4ba7-9b29-3f1151b46c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:15.762749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1092/2203] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:15.776844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481bde7-40ba-47e7-995f-253fda417f01 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:15.777360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2481bde7-40ba-47e7-995f-253fda417f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:15.861397 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481bde7-40ba-47e7-995f-253fda417f01 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:15.865363 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb315c-2798-4b48-b8e6-74edee014e39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:15.866060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb315c-2798-4b48-b8e6-74edee014e39 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:15.866379 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb315c-2798-4b48-b8e6-74edee014e39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:15.866658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb315c-2798-4b48-b8e6-74edee014e39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:15.877860 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb315c-2798-4b48-b8e6-74edee014e39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:15.879406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb315c-2798-4b48-b8e6-74edee014e39 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:15.902902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1112/2204] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:34:15.928251 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa8f77-f403-4ba7-9b29-3f1151b46c53 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:16.021864 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa8f77-f403-4ba7-9b29-3f1151b46c53 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:16.029497 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9632b-97db-4e63-aa3a-1eaddc981ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:16.030472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9632b-97db-4e63-aa3a-1eaddc981ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:16.030759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9632b-97db-4e63-aa3a-1eaddc981ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:16.031069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9632b-97db-4e63-aa3a-1eaddc981ac0 admin admin] PATH_INFO: `/v3/users/1c159a2e623d48269a8607de341a98f8` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:16.033102 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9632b-97db-4e63-aa3a-1eaddc981ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c159a2e623d48269a8607de341a98f8)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:16.035008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9632b-97db-4e63-aa3a-1eaddc981ac0 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:16.317851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e2b9632b-97db-4e63-aa3a-1eaddc981ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c159a2e623d48269a8607de341a98f8'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:16.438546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e2b9632b-97db-4e63-aa3a-1eaddc981ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c159a2e623d48269a8607de341a98f8'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:16.440438 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e2b9632b-97db-4e63-aa3a-1eaddc981ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c159a2e623d48269a8607de341a98f8'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:16.463566 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e2b9632b-97db-4e63-aa3a-1eaddc981ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c159a2e623d48269a8607de341a98f8'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:16.484124 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e2b9632b-97db-4e63-aa3a-1eaddc981ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c159a2e623d48269a8607de341a98f8'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:16.492703 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:16.499716 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1093/2205] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:34:15 2022] DELETE /identity/v3/users/1c159a2e623d48269a8607de341a98f8 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:16.501735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88afbd9-1982-4484-b48b-1c9eb90bec61 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:16.502538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88afbd9-1982-4484-b48b-1c9eb90bec61 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:16.502852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88afbd9-1982-4484-b48b-1c9eb90bec61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:16.534424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-d88afbd9-1982-4484-b48b-1c9eb90bec61 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:16.535621 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d88afbd9-1982-4484-b48b-1c9eb90bec61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:16.641011 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1113/2206] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:16.651442 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9632b-97db-4e63-aa3a-1eaddc981ac0 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:16.655338 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e2b9632b-97db-4e63-aa3a-1eaddc981ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:16.747454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9632b-97db-4e63-aa3a-1eaddc981ac0 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:16.758520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef53af-dab2-4cc0-b9a2-9ff3bccdc834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:16.759388 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef53af-dab2-4cc0-b9a2-9ff3bccdc834 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:16.759676 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef53af-dab2-4cc0-b9a2-9ff3bccdc834 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:16.759955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef53af-dab2-4cc0-b9a2-9ff3bccdc834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:16.772005 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef53af-dab2-4cc0-b9a2-9ff3bccdc834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:16.773644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef53af-dab2-4cc0-b9a2-9ff3bccdc834 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:16.790443 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1094/2207] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:17.555819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88afbd9-1982-4484-b48b-1c9eb90bec61 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:17.562048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d88afbd9-1982-4484-b48b-1c9eb90bec61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:17.584344 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88afbd9-1982-4484-b48b-1c9eb90bec61 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:17.603887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9832c-bdd3-4c2f-a61a-47139c405d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:17.605059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9832c-bdd3-4c2f-a61a-47139c405d63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:17.605478 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9832c-bdd3-4c2f-a61a-47139c405d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:17.605888 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9832c-bdd3-4c2f-a61a-47139c405d63 admin admin] PATH_INFO: `/v3/projects/e6f43de9b423400dab83b96e5b4038c9` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:17.607543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9832c-bdd3-4c2f-a61a-47139c405d63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6f43de9b423400dab83b96e5b4038c9)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:17.609793 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9832c-bdd3-4c2f-a61a-47139c405d63 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:17.789671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e3a9832c-bdd3-4c2f-a61a-47139c405d63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6f43de9b423400dab83b96e5b4038c9', 'request_id': 'req-e3a9832c-bdd3-4c2f-a61a-47139c405d63'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:17.893260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e3a9832c-bdd3-4c2f-a61a-47139c405d63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6f43de9b423400dab83b96e5b4038c9', 'request_id': 'req-e3a9832c-bdd3-4c2f-a61a-47139c405d63'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:17.899256 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:17.904343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e3a9832c-bdd3-4c2f-a61a-47139c405d63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6f43de9b423400dab83b96e5b4038c9', 'request_id': 'req-e3a9832c-bdd3-4c2f-a61a-47139c405d63'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:17.917127 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e3a9832c-bdd3-4c2f-a61a-47139c405d63 admin admin] The token cache is being invalidate because project e6f43de9b423400dab83b96e5b4038c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:34:17.917467 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e3a9832c-bdd3-4c2f-a61a-47139c405d63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:17.918147 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e3a9832c-bdd3-4c2f-a61a-47139c405d63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:17.920156 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1114/2208] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:34:17 2022] DELETE /identity/v3/projects/e6f43de9b423400dab83b96e5b4038c9 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:17.925620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef53af-dab2-4cc0-b9a2-9ff3bccdc834 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:17.944187 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef53af-dab2-4cc0-b9a2-9ff3bccdc834 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:18.012133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ab2e5-892e-479e-9d30-362d428cd54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:18.013897 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ab2e5-892e-479e-9d30-362d428cd54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:18.013897 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ab2e5-892e-479e-9d30-362d428cd54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:18.014243 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ab2e5-892e-479e-9d30-362d428cd54f admin admin] PATH_INFO: `/v3/users/2a102e6144574c378cf94cf1a8aaad51` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:18.015455 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ab2e5-892e-479e-9d30-362d428cd54f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a102e6144574c378cf94cf1a8aaad51)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:18.017998 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ab2e5-892e-479e-9d30-362d428cd54f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:18.308086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-420ab2e5-892e-479e-9d30-362d428cd54f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a102e6144574c378cf94cf1a8aaad51'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:18.309112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:18.414820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-420ab2e5-892e-479e-9d30-362d428cd54f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a102e6144574c378cf94cf1a8aaad51'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:18.416055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-420ab2e5-892e-479e-9d30-362d428cd54f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a102e6144574c378cf94cf1a8aaad51'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:18.446588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-420ab2e5-892e-479e-9d30-362d428cd54f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a102e6144574c378cf94cf1a8aaad51'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:18.485218 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-420ab2e5-892e-479e-9d30-362d428cd54f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a102e6144574c378cf94cf1a8aaad51'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:18.505897 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1095/2209] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:34:17 2022] DELETE /identity/v3/users/2a102e6144574c378cf94cf1a8aaad51 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:19.283223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9832c-bdd3-4c2f-a61a-47139c405d63 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:19.284676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e3a9832c-bdd3-4c2f-a61a-47139c405d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:19.321775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9832c-bdd3-4c2f-a61a-47139c405d63 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:19.446405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabfa5d-57ff-439d-a758-c138f54cde6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:19.447353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabfa5d-57ff-439d-a758-c138f54cde6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:19.448451 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabfa5d-57ff-439d-a758-c138f54cde6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:19.448798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabfa5d-57ff-439d-a758-c138f54cde6f admin admin] PATH_INFO: `/v3/projects/9fcddac2fa564af2b3e187ef740f6ba1` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:19.451487 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabfa5d-57ff-439d-a758-c138f54cde6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fcddac2fa564af2b3e187ef740f6ba1)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:19.454549 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabfa5d-57ff-439d-a758-c138f54cde6f admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:19.704446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-2fabfa5d-57ff-439d-a758-c138f54cde6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fcddac2fa564af2b3e187ef740f6ba1', 'request_id': 'req-2fabfa5d-57ff-439d-a758-c138f54cde6f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:19.856706 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-2fabfa5d-57ff-439d-a758-c138f54cde6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fcddac2fa564af2b3e187ef740f6ba1', 'request_id': 'req-2fabfa5d-57ff-439d-a758-c138f54cde6f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:19.877303 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-2fabfa5d-57ff-439d-a758-c138f54cde6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fcddac2fa564af2b3e187ef740f6ba1', 'request_id': 'req-2fabfa5d-57ff-439d-a758-c138f54cde6f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:19.920018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-2fabfa5d-57ff-439d-a758-c138f54cde6f admin admin] The token cache is being invalidate because project 9fcddac2fa564af2b3e187ef740f6ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:34:19.920386 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-2fabfa5d-57ff-439d-a758-c138f54cde6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:19.926501 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-2fabfa5d-57ff-439d-a758-c138f54cde6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:19.933544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1115/2210] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:34:19 2022] DELETE /identity/v3/projects/9fcddac2fa564af2b3e187ef740f6ba1 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:34:19.938973 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fdb02-e4f3-4127-8687-ba4ccb6b42cc None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:19.940201 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fdb02-e4f3-4127-8687-ba4ccb6b42cc None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:19.940733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fdb02-e4f3-4127-8687-ba4ccb6b42cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:19.971543 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-cc3fdb02-e4f3-4127-8687-ba4ccb6b42cc None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:19.979350 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-cc3fdb02-e4f3-4127-8687-ba4ccb6b42cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:20.158104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b44ba5-3bf2-4f83-816b-1f43bb69f6f0 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:20.158533 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b44ba5-3bf2-4f83-816b-1f43bb69f6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:20.168106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b44ba5-3bf2-4f83-816b-1f43bb69f6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:20.168713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1096/2211] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:20.205477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-a2b44ba5-3bf2-4f83-816b-1f43bb69f6f0 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:20.207094 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-a2b44ba5-3bf2-4f83-816b-1f43bb69f6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:20.226290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1116/2212] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:34:20.227904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fdb02-e4f3-4127-8687-ba4ccb6b42cc None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:20.248410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fdb02-e4f3-4127-8687-ba4ccb6b42cc None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:20.254848 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da222d4e-dab8-419f-b36d-d0c0415c8ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:20.255693 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da222d4e-dab8-419f-b36d-d0c0415c8ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:20.256054 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da222d4e-dab8-419f-b36d-d0c0415c8ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:20.256427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da222d4e-dab8-419f-b36d-d0c0415c8ab3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:20.257299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da222d4e-dab8-419f-b36d-d0c0415c8ab3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:34:20.258057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da222d4e-dab8-419f-b36d-d0c0415c8ab3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:20.259836 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da222d4e-dab8-419f-b36d-d0c0415c8ab3 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:20.277341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1097/2213] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:34:20.283253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b44ba5-3bf2-4f83-816b-1f43bb69f6f0 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:20.347613 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b44ba5-3bf2-4f83-816b-1f43bb69f6f0 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:20.358395 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267678e-9270-497f-813e-91a3eb627dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:20.360610 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267678e-9270-497f-813e-91a3eb627dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:20.362916 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267678e-9270-497f-813e-91a3eb627dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:20.363299 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267678e-9270-497f-813e-91a3eb627dc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:20.364919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267678e-9270-497f-813e-91a3eb627dc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:20.369376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267678e-9270-497f-813e-91a3eb627dc2 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:20.556072 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1117/2214] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:34:20 2022] POST /identity/v3/projects => generated 387 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:20.560444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da222d4e-dab8-419f-b36d-d0c0415c8ab3 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:20.583249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da222d4e-dab8-419f-b36d-d0c0415c8ab3 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:20.744795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e3042-35c4-4f6b-8250-92ee188b792f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:20.745675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e3042-35c4-4f6b-8250-92ee188b792f admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:20.746137 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e3042-35c4-4f6b-8250-92ee188b792f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:20.746618 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e3042-35c4-4f6b-8250-92ee188b792f admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:20.748084 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e3042-35c4-4f6b-8250-92ee188b792f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:20.752188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e3042-35c4-4f6b-8250-92ee188b792f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:20.873457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267678e-9270-497f-813e-91a3eb627dc2 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:20.875120 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1098/2215] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:34:20 2022] POST /identity/v3/users => generated 294 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:20.893478 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267678e-9270-497f-813e-91a3eb627dc2 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:20.901266 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c4376-1d01-4b7b-8eab-9b884d5c26a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:20.902282 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c4376-1d01-4b7b-8eab-9b884d5c26a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:20.903809 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c4376-1d01-4b7b-8eab-9b884d5c26a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:20.903809 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c4376-1d01-4b7b-8eab-9b884d5c26a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:20.904578 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c4376-1d01-4b7b-8eab-9b884d5c26a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:20.906401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c4376-1d01-4b7b-8eab-9b884d5c26a6 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:20.923287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1118/2216] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:20.929874 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e3042-35c4-4f6b-8250-92ee188b792f admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:20.942489 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e3042-35c4-4f6b-8250-92ee188b792f admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:20.949878 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81528409-e12c-4f0c-8dfe-2015f7b3b42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:20.955262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81528409-e12c-4f0c-8dfe-2015f7b3b42a admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:20.955262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81528409-e12c-4f0c-8dfe-2015f7b3b42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:20.955262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81528409-e12c-4f0c-8dfe-2015f7b3b42a admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:20.955262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81528409-e12c-4f0c-8dfe-2015f7b3b42a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:20.956216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81528409-e12c-4f0c-8dfe-2015f7b3b42a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:20.966990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1099/2217] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:20.971193 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c4376-1d01-4b7b-8eab-9b884d5c26a6 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:20.983898 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c4376-1d01-4b7b-8eab-9b884d5c26a6 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:20.989993 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3325b44-c532-4572-9800-87c4b6fcbfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:20.991124 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3325b44-c532-4572-9800-87c4b6fcbfba admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:20.997550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3325b44-c532-4572-9800-87c4b6fcbfba admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:20.997550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3325b44-c532-4572-9800-87c4b6fcbfba admin admin] PATH_INFO: `/v3/projects/33c91922070b4cf98a18b2d386611330/users/3806e400e09f44859b84f11eb9baa356/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:21.010107 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3325b44-c532-4572-9800-87c4b6fcbfba admin admin] RBAC: Authorizing `identity:create_grant(project_id=33c91922070b4cf98a18b2d386611330, user_id=3806e400e09f44859b84f11eb9baa356, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:21.016054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3325b44-c532-4572-9800-87c4b6fcbfba admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:21.130165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1119/2218] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:20 2022] PUT /identity/v3/projects/33c91922070b4cf98a18b2d386611330/users/3806e400e09f44859b84f11eb9baa356/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:21.136329 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57564ebf-7a94-420a-a6b8-e77a9762e2f0 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:21.136625 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57564ebf-7a94-420a-a6b8-e77a9762e2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:21.136909 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57564ebf-7a94-420a-a6b8-e77a9762e2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:21.167818 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-57564ebf-7a94-420a-a6b8-e77a9762e2f0 None None] MFA Rules not processed for user `3806e400e09f44859b84f11eb9baa356`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:21.169072 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-57564ebf-7a94-420a-a6b8-e77a9762e2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:21.194539 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:21.259910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1100/2219] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:34:21.260944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3325b44-c532-4572-9800-87c4b6fcbfba admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:21.279546 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:21.361800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3325b44-c532-4572-9800-87c4b6fcbfba admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:21.368591 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcaee6b-c328-48b3-8e13-d6dfbef4cef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:21.369643 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcaee6b-c328-48b3-8e13-d6dfbef4cef7 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:21.370104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcaee6b-c328-48b3-8e13-d6dfbef4cef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:21.370953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcaee6b-c328-48b3-8e13-d6dfbef4cef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:21.372465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcaee6b-c328-48b3-8e13-d6dfbef4cef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:21.374706 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcaee6b-c328-48b3-8e13-d6dfbef4cef7 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:21.456174 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1120/2220] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:34:21 2022] POST /identity/v3/projects => generated 383 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:34:21.461861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57564ebf-7a94-420a-a6b8-e77a9762e2f0 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:21.475156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57564ebf-7a94-420a-a6b8-e77a9762e2f0 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:21.485000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a670c3-aa80-4013-bf4f-b19cfc9f61c7 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:21.485291 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a670c3-aa80-4013-bf4f-b19cfc9f61c7 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:21.485575 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a670c3-aa80-4013-bf4f-b19cfc9f61c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:21.519524 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-44a670c3-aa80-4013-bf4f-b19cfc9f61c7 None None] MFA Rules not processed for user `dce92cce74c84abcbec76358c860535f`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:21.520871 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-44a670c3-aa80-4013-bf4f-b19cfc9f61c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:21.585054 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbbc5f-79ba-4462-bf41-bcab8d02b198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:21.587907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbbc5f-79ba-4462-bf41-bcab8d02b198 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:21.587907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbbc5f-79ba-4462-bf41-bcab8d02b198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:21.587907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbbc5f-79ba-4462-bf41-bcab8d02b198 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:21.588368 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbbc5f-79ba-4462-bf41-bcab8d02b198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:21.590626 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbbc5f-79ba-4462-bf41-bcab8d02b198 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:21.620373 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1121/2221] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:21.625106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a670c3-aa80-4013-bf4f-b19cfc9f61c7 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:21.626525 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-44a670c3-aa80-4013-bf4f-b19cfc9f61c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:21.641836 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a670c3-aa80-4013-bf4f-b19cfc9f61c7 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:21.646764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a7f86-7b41-48ef-b7d8-ce89e7e692ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:21.647441 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a7f86-7b41-48ef-b7d8-ce89e7e692ee admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:21.647728 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a7f86-7b41-48ef-b7d8-ce89e7e692ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:21.648009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a7f86-7b41-48ef-b7d8-ce89e7e692ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:21.648711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a7f86-7b41-48ef-b7d8-ce89e7e692ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:21.650774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a7f86-7b41-48ef-b7d8-ce89e7e692ee admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:21.741635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1101/2222] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:34:21 2022] POST /identity/v3/users => generated 298 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:21.742907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbbc5f-79ba-4462-bf41-bcab8d02b198 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:21.755305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbbc5f-79ba-4462-bf41-bcab8d02b198 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:21.760461 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1965f00-5e45-485e-aa66-154e4b40f740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:21.761413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1965f00-5e45-485e-aa66-154e4b40f740 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:21.761857 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1965f00-5e45-485e-aa66-154e4b40f740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:21.762346 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1965f00-5e45-485e-aa66-154e4b40f740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:21.766838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1122/2223] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:34:21 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:34:21.767714 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1965f00-5e45-485e-aa66-154e4b40f740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:21.768859 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a7f86-7b41-48ef-b7d8-ce89e7e692ee admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:21.770214 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1965f00-5e45-485e-aa66-154e4b40f740 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:21.799620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1102/2224] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:21.807327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1965f00-5e45-485e-aa66-154e4b40f740 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:21.830199 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1965f00-5e45-485e-aa66-154e4b40f740 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:21.944635 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a7f86-7b41-48ef-b7d8-ce89e7e692ee admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:21.956574 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c74a1f-4d7d-4895-b39e-834259d3d47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:21.957344 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c74a1f-4d7d-4895-b39e-834259d3d47e admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:21.957826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c74a1f-4d7d-4895-b39e-834259d3d47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:21.958119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c74a1f-4d7d-4895-b39e-834259d3d47e admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:21.959514 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c74a1f-4d7d-4895-b39e-834259d3d47e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:21.961857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c74a1f-4d7d-4895-b39e-834259d3d47e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:21.992044 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b157ce-da60-41c5-aea1-454f0aadf3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:21.994673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b157ce-da60-41c5-aea1-454f0aadf3aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:21.996475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b157ce-da60-41c5-aea1-454f0aadf3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:22.004747 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b157ce-da60-41c5-aea1-454f0aadf3aa admin admin] PATH_INFO: `/v3/projects/82f1721d93e54dbc8b30c4bea4b9e82f/users/6bdf500153254122b16c7d3e5eb441c8/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:22.026609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b157ce-da60-41c5-aea1-454f0aadf3aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=82f1721d93e54dbc8b30c4bea4b9e82f, user_id=6bdf500153254122b16c7d3e5eb441c8, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:22.031542 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b157ce-da60-41c5-aea1-454f0aadf3aa admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:22.115937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1123/2225] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:34:21 2022] POST /identity/v3/users => generated 299 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:22.120235 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c74a1f-4d7d-4895-b39e-834259d3d47e admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:22.145051 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1103/2226] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:21 2022] PUT /identity/v3/projects/82f1721d93e54dbc8b30c4bea4b9e82f/users/6bdf500153254122b16c7d3e5eb441c8/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:22.146236 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b157ce-da60-41c5-aea1-454f0aadf3aa admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:22.150844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c74a1f-4d7d-4895-b39e-834259d3d47e admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:22.281057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b157ce-da60-41c5-aea1-454f0aadf3aa admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:22.292963 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec944e-6756-47fa-94be-c9f1a69f23c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:22.294627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec944e-6756-47fa-94be-c9f1a69f23c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:22.295029 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec944e-6756-47fa-94be-c9f1a69f23c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:22.295428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec944e-6756-47fa-94be-c9f1a69f23c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:22.296607 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec944e-6756-47fa-94be-c9f1a69f23c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:22.298794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec944e-6756-47fa-94be-c9f1a69f23c0 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:22.329103 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1104/2227] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:22.336289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec944e-6756-47fa-94be-c9f1a69f23c0 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:22.358216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec944e-6756-47fa-94be-c9f1a69f23c0 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:22.376531 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b980a53-74bd-44e4-8bc8-1ae3b297b578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:22.377766 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b980a53-74bd-44e4-8bc8-1ae3b297b578 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:22.377905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b980a53-74bd-44e4-8bc8-1ae3b297b578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:22.378313 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b980a53-74bd-44e4-8bc8-1ae3b297b578 admin admin] PATH_INFO: `/v3/projects/82f1721d93e54dbc8b30c4bea4b9e82f/users/6bdf500153254122b16c7d3e5eb441c8/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:22.387370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b980a53-74bd-44e4-8bc8-1ae3b297b578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82f1721d93e54dbc8b30c4bea4b9e82f, user_id=6bdf500153254122b16c7d3e5eb441c8, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:22.387370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b980a53-74bd-44e4-8bc8-1ae3b297b578 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:22.425008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1105/2228] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:22 2022] PUT /identity/v3/projects/82f1721d93e54dbc8b30c4bea4b9e82f/users/6bdf500153254122b16c7d3e5eb441c8/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:22.438353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fd4d8-bf48-4af0-9667-0e8ae19f92ee None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:22.438685 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fd4d8-bf48-4af0-9667-0e8ae19f92ee None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:22.438993 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fd4d8-bf48-4af0-9667-0e8ae19f92ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:22.441952 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06ea8b-33cb-41e2-834e-154020a915fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:22.442921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06ea8b-33cb-41e2-834e-154020a915fd admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:22.443208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06ea8b-33cb-41e2-834e-154020a915fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:22.443719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06ea8b-33cb-41e2-834e-154020a915fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:22.444466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06ea8b-33cb-41e2-834e-154020a915fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:22.446442 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06ea8b-33cb-41e2-834e-154020a915fd admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:22.463774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1124/2229] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:34:22.469062 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06ea8b-33cb-41e2-834e-154020a915fd admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:22.485492 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06ea8b-33cb-41e2-834e-154020a915fd admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:22.486927 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-143fd4d8-bf48-4af0-9667-0e8ae19f92ee None None] MFA Rules not processed for user `6bdf500153254122b16c7d3e5eb441c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:22.492109 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-143fd4d8-bf48-4af0-9667-0e8ae19f92ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:22.578554 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c0862-8408-41a4-b58a-ed24d04e6f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:22.580050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c0862-8408-41a4-b58a-ed24d04e6f37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:22.580356 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c0862-8408-41a4-b58a-ed24d04e6f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:22.580712 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c0862-8408-41a4-b58a-ed24d04e6f37 admin admin] PATH_INFO: `/v3/projects/d33d090d45b04ef99ceae6dd4f806320/users/d8a92f26477b40dcaf77913e55c5fa6b/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:22.592819 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1106/2230] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:22.605020 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fd4d8-bf48-4af0-9667-0e8ae19f92ee None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:22.608716 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-143fd4d8-bf48-4af0-9667-0e8ae19f92ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:22.619792 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c0862-8408-41a4-b58a-ed24d04e6f37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d33d090d45b04ef99ceae6dd4f806320, user_id=d8a92f26477b40dcaf77913e55c5fa6b, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:22.624032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c0862-8408-41a4-b58a-ed24d04e6f37 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:22.699013 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fd4d8-bf48-4af0-9667-0e8ae19f92ee None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:22.707141 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d37c8-fcff-4f31-ba83-c2d288e2e7c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:22.708357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d37c8-fcff-4f31-ba83-c2d288e2e7c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:22.708923 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d37c8-fcff-4f31-ba83-c2d288e2e7c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:22.714379 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d37c8-fcff-4f31-ba83-c2d288e2e7c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:22.734447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d37c8-fcff-4f31-ba83-c2d288e2e7c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:22.741314 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d37c8-fcff-4f31-ba83-c2d288e2e7c7 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:22.746520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1125/2231] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:22 2022] PUT /identity/v3/projects/d33d090d45b04ef99ceae6dd4f806320/users/d8a92f26477b40dcaf77913e55c5fa6b/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:34:22.753238 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c0862-8408-41a4-b58a-ed24d04e6f37 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:22.759426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1107/2232] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:22.855828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c0862-8408-41a4-b58a-ed24d04e6f37 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:22.860864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964782e-b2fe-43f6-bd08-da02a7d51580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:22.861503 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964782e-b2fe-43f6-bd08-da02a7d51580 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:22.861790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964782e-b2fe-43f6-bd08-da02a7d51580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:22.864541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964782e-b2fe-43f6-bd08-da02a7d51580 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:22.864541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964782e-b2fe-43f6-bd08-da02a7d51580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:22.864541 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964782e-b2fe-43f6-bd08-da02a7d51580 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:22.875210 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1126/2233] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:22.879494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d37c8-fcff-4f31-ba83-c2d288e2e7c7 service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:22.891302 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d37c8-fcff-4f31-ba83-c2d288e2e7c7 service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:22.896459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137207b-c3d4-48dd-93d7-36486a105ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:22.897355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137207b-c3d4-48dd-93d7-36486a105ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:22.897636 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137207b-c3d4-48dd-93d7-36486a105ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:22.897911 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137207b-c3d4-48dd-93d7-36486a105ec4 admin admin] PATH_INFO: `/v3/projects/d33d090d45b04ef99ceae6dd4f806320/users/d8a92f26477b40dcaf77913e55c5fa6b/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:22.899934 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137207b-c3d4-48dd-93d7-36486a105ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d33d090d45b04ef99ceae6dd4f806320, user_id=d8a92f26477b40dcaf77913e55c5fa6b, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:22.902623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137207b-c3d4-48dd-93d7-36486a105ec4 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:22.930992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1108/2234] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:22 2022] PUT /identity/v3/projects/d33d090d45b04ef99ceae6dd4f806320/users/d8a92f26477b40dcaf77913e55c5fa6b/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:24.626787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:25.067735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:26.694226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964782e-b2fe-43f6-bd08-da02a7d51580 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:26.714585 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964782e-b2fe-43f6-bd08-da02a7d51580 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:26.850868 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2e339-6cc4-4a8f-b321-4000314e88c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:26.851939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2e339-6cc4-4a8f-b321-4000314e88c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:26.852437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2e339-6cc4-4a8f-b321-4000314e88c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:26.852772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2e339-6cc4-4a8f-b321-4000314e88c0 admin admin] PATH_INFO: `/v3/users/3806e400e09f44859b84f11eb9baa356` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:26.861336 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2e339-6cc4-4a8f-b321-4000314e88c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3806e400e09f44859b84f11eb9baa356)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:26.863340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2e339-6cc4-4a8f-b321-4000314e88c0 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:27.110047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-8de2e339-6cc4-4a8f-b321-4000314e88c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3806e400e09f44859b84f11eb9baa356'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:27.276698 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-8de2e339-6cc4-4a8f-b321-4000314e88c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3806e400e09f44859b84f11eb9baa356'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:27.300071 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-8de2e339-6cc4-4a8f-b321-4000314e88c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3806e400e09f44859b84f11eb9baa356'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:27.300915 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-8de2e339-6cc4-4a8f-b321-4000314e88c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3806e400e09f44859b84f11eb9baa356'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:27.328576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-8de2e339-6cc4-4a8f-b321-4000314e88c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3806e400e09f44859b84f11eb9baa356'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:27.357198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2049214-777a-4002-9805-4cd0ef41a210 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:27.357872 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2049214-777a-4002-9805-4cd0ef41a210 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:27.358228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2049214-777a-4002-9805-4cd0ef41a210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:27.367709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1127/2235] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:34:26 2022] DELETE /identity/v3/users/3806e400e09f44859b84f11eb9baa356 => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:27.383397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-a2049214-777a-4002-9805-4cd0ef41a210 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:27.385314 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a2049214-777a-4002-9805-4cd0ef41a210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:27.419220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:27.454708 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:27.554322 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1109/2236] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:27.573696 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2e339-6cc4-4a8f-b321-4000314e88c0 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:27.577346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8de2e339-6cc4-4a8f-b321-4000314e88c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:27.709347 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2e339-6cc4-4a8f-b321-4000314e88c0 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:27.714546 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e0cd1-20fd-472e-a213-31c3cf215905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:27.715226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e0cd1-20fd-472e-a213-31c3cf215905 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:27.715543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e0cd1-20fd-472e-a213-31c3cf215905 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:27.716463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e0cd1-20fd-472e-a213-31c3cf215905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:27.728619 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e0cd1-20fd-472e-a213-31c3cf215905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:27.730473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e0cd1-20fd-472e-a213-31c3cf215905 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:27.743260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1128/2237] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:28.759501 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2049214-777a-4002-9805-4cd0ef41a210 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:28.762579 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a2049214-777a-4002-9805-4cd0ef41a210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:28.783359 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2049214-777a-4002-9805-4cd0ef41a210 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:28.792894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:28.794769 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:28.795301 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:28.796892 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431 admin admin] PATH_INFO: `/v3/projects/33c91922070b4cf98a18b2d386611330` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:28.801536 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33c91922070b4cf98a18b2d386611330)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:28.804026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:28.988122 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33c91922070b4cf98a18b2d386611330', 'request_id': 'req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:29.094756 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33c91922070b4cf98a18b2d386611330', 'request_id': 'req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:29.105364 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33c91922070b4cf98a18b2d386611330', 'request_id': 'req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:29.118226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431 admin admin] The token cache is being invalidate because project 33c91922070b4cf98a18b2d386611330 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:34:29.118480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:29.119169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:29.120755 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1110/2238] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:34:28 2022] DELETE /identity/v3/projects/33c91922070b4cf98a18b2d386611330 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:29.126032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e0cd1-20fd-472e-a213-31c3cf215905 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:29.155973 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e0cd1-20fd-472e-a213-31c3cf215905 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:29.312545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59414f79-cc48-43d9-80b3-d7eb50c47372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:29.313810 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59414f79-cc48-43d9-80b3-d7eb50c47372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:29.314294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59414f79-cc48-43d9-80b3-d7eb50c47372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:29.314751 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59414f79-cc48-43d9-80b3-d7eb50c47372 admin admin] PATH_INFO: `/v3/users/6bdf500153254122b16c7d3e5eb441c8` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:29.316995 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59414f79-cc48-43d9-80b3-d7eb50c47372 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bdf500153254122b16c7d3e5eb441c8)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:29.319317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59414f79-cc48-43d9-80b3-d7eb50c47372 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:29.729521 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-59414f79-cc48-43d9-80b3-d7eb50c47372 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bdf500153254122b16c7d3e5eb441c8'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:29.831816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-59414f79-cc48-43d9-80b3-d7eb50c47372 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bdf500153254122b16c7d3e5eb441c8'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:29.852832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-59414f79-cc48-43d9-80b3-d7eb50c47372 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bdf500153254122b16c7d3e5eb441c8'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:29.855425 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-59414f79-cc48-43d9-80b3-d7eb50c47372 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bdf500153254122b16c7d3e5eb441c8'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:29.871423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-59414f79-cc48-43d9-80b3-d7eb50c47372 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bdf500153254122b16c7d3e5eb441c8'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:29.884831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1129/2239] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:34:29 2022] DELETE /identity/v3/users/6bdf500153254122b16c7d3e5eb441c8 => generated 0 bytes in 755 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:29.913515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:30.438866 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:30.440002 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:30.461153 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69da8e-d59e-40b8-84b7-c1d7f1a55431 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:30.603378 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77275284-0f42-466b-8797-9cb430df3cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:30.604923 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77275284-0f42-466b-8797-9cb430df3cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:30.605401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77275284-0f42-466b-8797-9cb430df3cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:30.605825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77275284-0f42-466b-8797-9cb430df3cd4 admin admin] PATH_INFO: `/v3/projects/82f1721d93e54dbc8b30c4bea4b9e82f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:30.611055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77275284-0f42-466b-8797-9cb430df3cd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82f1721d93e54dbc8b30c4bea4b9e82f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:30.614140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77275284-0f42-466b-8797-9cb430df3cd4 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:30.755600 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:30.758657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-77275284-0f42-466b-8797-9cb430df3cd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82f1721d93e54dbc8b30c4bea4b9e82f', 'request_id': 'req-77275284-0f42-466b-8797-9cb430df3cd4'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:30.824278 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-77275284-0f42-466b-8797-9cb430df3cd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82f1721d93e54dbc8b30c4bea4b9e82f', 'request_id': 'req-77275284-0f42-466b-8797-9cb430df3cd4'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:30.832266 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-77275284-0f42-466b-8797-9cb430df3cd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82f1721d93e54dbc8b30c4bea4b9e82f', 'request_id': 'req-77275284-0f42-466b-8797-9cb430df3cd4'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:30.848892 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-77275284-0f42-466b-8797-9cb430df3cd4 admin admin] The token cache is being invalidate because project 82f1721d93e54dbc8b30c4bea4b9e82f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:34:30.849183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-77275284-0f42-466b-8797-9cb430df3cd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:30.850698 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-77275284-0f42-466b-8797-9cb430df3cd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:30.852340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1111/2240] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:34:30 2022] DELETE /identity/v3/projects/82f1721d93e54dbc8b30c4bea4b9e82f => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:30.858676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddf16d-b690-46d5-b91a-4992340106b3 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:30.859568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddf16d-b690-46d5-b91a-4992340106b3 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:30.860173 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddf16d-b690-46d5-b91a-4992340106b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:30.887985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-c0ddf16d-b690-46d5-b91a-4992340106b3 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:30.890947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c0ddf16d-b690-46d5-b91a-4992340106b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:31.029852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1130/2241] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:31.035570 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496ee71-228d-48d4-93b3-2439a1e06cb8 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:31.036462 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496ee71-228d-48d4-93b3-2439a1e06cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:31.036752 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496ee71-228d-48d4-93b3-2439a1e06cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:31.080871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-0496ee71-228d-48d4-93b3-2439a1e06cb8 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:31.082101 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0496ee71-228d-48d4-93b3-2439a1e06cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:31.105582 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1112/2242] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:31.110979 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddf16d-b690-46d5-b91a-4992340106b3 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:31.128506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddf16d-b690-46d5-b91a-4992340106b3 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:31.134317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32f6cc-c86b-4595-887a-400055d0991c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:31.135491 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32f6cc-c86b-4595-887a-400055d0991c admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:31.135932 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32f6cc-c86b-4595-887a-400055d0991c admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:31.136388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32f6cc-c86b-4595-887a-400055d0991c admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:31.138449 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32f6cc-c86b-4595-887a-400055d0991c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:34:31.138945 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32f6cc-c86b-4595-887a-400055d0991c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:31.141177 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32f6cc-c86b-4595-887a-400055d0991c admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:31.163488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1131/2243] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:34:31.167185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496ee71-228d-48d4-93b3-2439a1e06cb8 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:31.181472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496ee71-228d-48d4-93b3-2439a1e06cb8 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:31.195823 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ce486-e03d-4dbb-bd96-bcfe7acf4da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:31.196505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ce486-e03d-4dbb-bd96-bcfe7acf4da2 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:31.198409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ce486-e03d-4dbb-bd96-bcfe7acf4da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:31.198409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ce486-e03d-4dbb-bd96-bcfe7acf4da2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:31.198409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ce486-e03d-4dbb-bd96-bcfe7acf4da2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:31.209151 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ce486-e03d-4dbb-bd96-bcfe7acf4da2 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:31.294191 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a054ad-fa4f-43ef-8294-cb333a8c41c8 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:31.294526 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a054ad-fa4f-43ef-8294-cb333a8c41c8 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:31.294815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a054ad-fa4f-43ef-8294-cb333a8c41c8 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:31.296559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1132/2244] 10.222.0.78 () {54 vars in 921 bytes} [Wed Jul 20 15:34:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:34:31.313381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcac73-d046-4850-9f60-7991b49925f9 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:31.313674 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcac73-d046-4850-9f60-7991b49925f9 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:31.314365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcac73-d046-4850-9f60-7991b49925f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:31.335282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1113/2245] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:34:31 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:31.336949 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ce486-e03d-4dbb-bd96-bcfe7acf4da2 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:31.357677 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6ebcac73-d046-4850-9f60-7991b49925f9 None None] MFA Rules not processed for user `2192547babde4449a46f9cb37ed64524`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:31.359038 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6ebcac73-d046-4850-9f60-7991b49925f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:31.463475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ce486-e03d-4dbb-bd96-bcfe7acf4da2 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:31.486676 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3f69d-d8f9-4d72-92b8-2287774ea668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:31.486676 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3f69d-d8f9-4d72-92b8-2287774ea668 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:31.486676 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3f69d-d8f9-4d72-92b8-2287774ea668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:31.486676 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3f69d-d8f9-4d72-92b8-2287774ea668 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:31.487190 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3f69d-d8f9-4d72-92b8-2287774ea668 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:31.495788 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3f69d-d8f9-4d72-92b8-2287774ea668 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:31.512381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1133/2246] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Jul 20 15:34:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:31.513520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcac73-d046-4850-9f60-7991b49925f9 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:31.519715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6ebcac73-d046-4850-9f60-7991b49925f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:31.536780 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcac73-d046-4850-9f60-7991b49925f9 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:31.551461 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e9260-6b3d-447a-adc1-ff34d55302ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:31.553789 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e9260-6b3d-447a-adc1-ff34d55302ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:31.554224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e9260-6b3d-447a-adc1-ff34d55302ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:31.554803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e9260-6b3d-447a-adc1-ff34d55302ee admin admin] PATH_INFO: `/v3/users/bbc3a225e07c4cef9f6b9713fc85942c` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:31.559492 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e9260-6b3d-447a-adc1-ff34d55302ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbc3a225e07c4cef9f6b9713fc85942c)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:31.567823 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e9260-6b3d-447a-adc1-ff34d55302ee admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:31.626784 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1114/2247] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:34:31 2022] POST /identity/v3/users => generated 290 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:31.631201 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3f69d-d8f9-4d72-92b8-2287774ea668 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:31.647943 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3f69d-d8f9-4d72-92b8-2287774ea668 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:31.656154 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7f997-c677-46c5-ad18-b62eb8ed17a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:31.656984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7f997-c677-46c5-ad18-b62eb8ed17a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:31.658046 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7f997-c677-46c5-ad18-b62eb8ed17a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:31.658604 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7f997-c677-46c5-ad18-b62eb8ed17a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:31.659460 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7f997-c677-46c5-ad18-b62eb8ed17a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:31.662426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7f997-c677-46c5-ad18-b62eb8ed17a6 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:31.677827 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1115/2248] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:34:31.683557 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7f997-c677-46c5-ad18-b62eb8ed17a6 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:31.699714 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7f997-c677-46c5-ad18-b62eb8ed17a6 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:31.707135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa28de-7f85-488c-9439-85d7b219e244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:31.707971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa28de-7f85-488c-9439-85d7b219e244 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:31.708329 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa28de-7f85-488c-9439-85d7b219e244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:31.708681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa28de-7f85-488c-9439-85d7b219e244 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:31.709871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa28de-7f85-488c-9439-85d7b219e244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:31.711903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa28de-7f85-488c-9439-85d7b219e244 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:31.730697 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1116/2249] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:31.732818 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa28de-7f85-488c-9439-85d7b219e244 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:31.764497 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa28de-7f85-488c-9439-85d7b219e244 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:31.771875 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cb5c3-2bfb-4af1-ab9f-15680cf9b59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:31.772819 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cb5c3-2bfb-4af1-ab9f-15680cf9b59b admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:31.773108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cb5c3-2bfb-4af1-ab9f-15680cf9b59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:31.773396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cb5c3-2bfb-4af1-ab9f-15680cf9b59b admin admin] PATH_INFO: `/v3/projects/2139071aef014b79a1737bd9fadd3db9/users/87bdbc4bad434d7ea9273466532ee7e1/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:31.809283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-db4e9260-6b3d-447a-adc1-ff34d55302ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbc3a225e07c4cef9f6b9713fc85942c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:31.825975 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76cb5c3-2bfb-4af1-ab9f-15680cf9b59b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2139071aef014b79a1737bd9fadd3db9, user_id=87bdbc4bad434d7ea9273466532ee7e1, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:31.826615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76cb5c3-2bfb-4af1-ab9f-15680cf9b59b admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:31.927655 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-db4e9260-6b3d-447a-adc1-ff34d55302ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbc3a225e07c4cef9f6b9713fc85942c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:31.944058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-db4e9260-6b3d-447a-adc1-ff34d55302ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbc3a225e07c4cef9f6b9713fc85942c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:31.944795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1117/2250] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:31 2022] PUT /identity/v3/projects/2139071aef014b79a1737bd9fadd3db9/users/87bdbc4bad434d7ea9273466532ee7e1/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:31.945773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cb5c3-2bfb-4af1-ab9f-15680cf9b59b admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:31.947039 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-db4e9260-6b3d-447a-adc1-ff34d55302ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbc3a225e07c4cef9f6b9713fc85942c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:31.957483 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e76cb5c3-2bfb-4af1-ab9f-15680cf9b59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:31.977692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-db4e9260-6b3d-447a-adc1-ff34d55302ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbc3a225e07c4cef9f6b9713fc85942c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:31.987490 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1134/2251] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:34:31 2022] DELETE /identity/v3/users/bbc3a225e07c4cef9f6b9713fc85942c => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:31.990336 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49c32f-3009-4340-ac17-60f426c70b74 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:31.991258 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49c32f-3009-4340-ac17-60f426c70b74 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:31.995524 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49c32f-3009-4340-ac17-60f426c70b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:32.038201 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-de49c32f-3009-4340-ac17-60f426c70b74 None None] MFA Rules not processed for user `87bdbc4bad434d7ea9273466532ee7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:32.039715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-de49c32f-3009-4340-ac17-60f426c70b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:32.072932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cb5c3-2bfb-4af1-ab9f-15680cf9b59b admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:32.171764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1135/2252] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:32.174224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:32.178783 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49c32f-3009-4340-ac17-60f426c70b74 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:32.194428 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-de49c32f-3009-4340-ac17-60f426c70b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:32.262010 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016103b3-0e04-4ffd-ac82-43f737275123 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2192547babde4449a46f9cb37ed64524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:32.263141 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016103b3-0e04-4ffd-ac82-43f737275123 service placement] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:32.263645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016103b3-0e04-4ffd-ac82-43f737275123 service placement] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:32.264108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016103b3-0e04-4ffd-ac82-43f737275123 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:32.269659 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-016103b3-0e04-4ffd-ac82-43f737275123 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:32.287697 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016103b3-0e04-4ffd-ac82-43f737275123 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:32.289621 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016103b3-0e04-4ffd-ac82-43f737275123 service placement] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:32.302954 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1118/2253] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:32.312526 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49c32f-3009-4340-ac17-60f426c70b74 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:32.318293 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd8028-ad04-4eec-bdc3-ec75669fa16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:32.319587 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd8028-ad04-4eec-bdc3-ec75669fa16d admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:32.319927 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd8028-ad04-4eec-bdc3-ec75669fa16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:32.320203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd8028-ad04-4eec-bdc3-ec75669fa16d admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:32.321142 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd8028-ad04-4eec-bdc3-ec75669fa16d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:32.324530 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd8028-ad04-4eec-bdc3-ec75669fa16d admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:32.392024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:DELETE=4,keystone:SELECT=90,keystone:INSERT=6 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:32.442797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1136/2254] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:34:32 2022] POST /identity/v3/projects => generated 381 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:32.444883 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016103b3-0e04-4ffd-ac82-43f737275123 service placement] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:32.457659 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016103b3-0e04-4ffd-ac82-43f737275123 service placement] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:32.515226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd8028-ad04-4eec-bdc3-ec75669fa16d admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:32.520748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f4dd8028-ad04-4eec-bdc3-ec75669fa16d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:32.601537 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c369b7-380d-43c3-a428-7ea4b0b5a2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:32.602429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c369b7-380d-43c3-a428-7ea4b0b5a2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:32.607140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c369b7-380d-43c3-a428-7ea4b0b5a2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:32.607140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c369b7-380d-43c3-a428-7ea4b0b5a2d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:32.607140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c369b7-380d-43c3-a428-7ea4b0b5a2d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:32.607140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c369b7-380d-43c3-a428-7ea4b0b5a2d2 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:32.674218 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd8028-ad04-4eec-bdc3-ec75669fa16d admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:32.680664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:32.681556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:32.681834 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:32.682108 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97 admin admin] PATH_INFO: `/v3/projects/1cd8d3e7499342d6820afd0c022033d7` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:32.683513 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd8d3e7499342d6820afd0c022033d7)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:32.685475 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:32.763394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1119/2255] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:34:32 2022] POST /identity/v3/users => generated 297 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:32.768623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c369b7-380d-43c3-a428-7ea4b0b5a2d2 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:32.797475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c369b7-380d-43c3-a428-7ea4b0b5a2d2 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:32.804243 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786287f-56e2-4bac-94a0-ab77dfceb5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:32.804613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786287f-56e2-4bac-94a0-ab77dfceb5ff admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:32.804998 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786287f-56e2-4bac-94a0-ab77dfceb5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:32.805708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786287f-56e2-4bac-94a0-ab77dfceb5ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:32.806998 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786287f-56e2-4bac-94a0-ab77dfceb5ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:32.809822 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786287f-56e2-4bac-94a0-ab77dfceb5ff admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:32.836231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1120/2256] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:32.837813 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786287f-56e2-4bac-94a0-ab77dfceb5ff admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:32.861840 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786287f-56e2-4bac-94a0-ab77dfceb5ff admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:32.877050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd8d3e7499342d6820afd0c022033d7', 'request_id': 'req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:32.947830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd8d3e7499342d6820afd0c022033d7', 'request_id': 'req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:32.957362 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd8d3e7499342d6820afd0c022033d7', 'request_id': 'req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:32.977032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97 admin admin] The token cache is being invalidate because project 1cd8d3e7499342d6820afd0c022033d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:34:32.977778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:32.979392 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:32.998489 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7a081-4452-43e8-9a68-82cd88e8c127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:32.999236 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1137/2257] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:34:32 2022] DELETE /identity/v3/projects/1cd8d3e7499342d6820afd0c022033d7 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:33.000147 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7a081-4452-43e8-9a68-82cd88e8c127 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:33.000639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7a081-4452-43e8-9a68-82cd88e8c127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:33.000946 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7a081-4452-43e8-9a68-82cd88e8c127 admin admin] PATH_INFO: `/v3/projects/7e699ac2f7b14177a7a41f3b9ea2beaa/users/96a007e611eb4be3b9bd185be15e4112/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:33.003222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:33.004514 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:33.026602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eeeb57-b242-41f1-b55b-ac2c3fa9ca97 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:33.039715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7a081-4452-43e8-9a68-82cd88e8c127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e699ac2f7b14177a7a41f3b9ea2beaa, user_id=96a007e611eb4be3b9bd185be15e4112, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:33.040741 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4a3cc-bccc-4421-bd7c-598a50286e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:33.042863 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7a081-4452-43e8-9a68-82cd88e8c127 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:33.043031 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4a3cc-bccc-4421-bd7c-598a50286e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:33.044473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4a3cc-bccc-4421-bd7c-598a50286e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:33.044756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4a3cc-bccc-4421-bd7c-598a50286e84 admin admin] PATH_INFO: `/v3/users/389c21cefe094be18841369f0f1b8570` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:33.046893 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4a3cc-bccc-4421-bd7c-598a50286e84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=389c21cefe094be18841369f0f1b8570)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:33.050664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4a3cc-bccc-4421-bd7c-598a50286e84 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:33.270713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1121/2258] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:32 2022] PUT /identity/v3/projects/7e699ac2f7b14177a7a41f3b9ea2beaa/users/96a007e611eb4be3b9bd185be15e4112/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:33.279284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7a081-4452-43e8-9a68-82cd88e8c127 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:33.281834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-38d7a081-4452-43e8-9a68-82cd88e8c127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:33.392338 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7a081-4452-43e8-9a68-82cd88e8c127 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:33.397658 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba09e3-82d5-4a0e-aa97-8b742532bed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:33.399241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba09e3-82d5-4a0e-aa97-8b742532bed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:33.400274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba09e3-82d5-4a0e-aa97-8b742532bed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:33.400614 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba09e3-82d5-4a0e-aa97-8b742532bed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:33.401512 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba09e3-82d5-4a0e-aa97-8b742532bed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:33.407638 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba09e3-82d5-4a0e-aa97-8b742532bed5 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:33.431544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1122/2259] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:34:33.432536 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba09e3-82d5-4a0e-aa97-8b742532bed5 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:33.467134 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba09e3-82d5-4a0e-aa97-8b742532bed5 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:33.492845 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-feb4a3cc-bccc-4421-bd7c-598a50286e84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '389c21cefe094be18841369f0f1b8570'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:33.493999 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001f2af-450b-41a5-b73c-c4c906d9d59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:33.494744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001f2af-450b-41a5-b73c-c4c906d9d59f admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:33.495030 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001f2af-450b-41a5-b73c-c4c906d9d59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:33.495308 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001f2af-450b-41a5-b73c-c4c906d9d59f admin admin] PATH_INFO: `/v3/projects/7e699ac2f7b14177a7a41f3b9ea2beaa/users/96a007e611eb4be3b9bd185be15e4112/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:33.500291 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001f2af-450b-41a5-b73c-c4c906d9d59f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e699ac2f7b14177a7a41f3b9ea2beaa, user_id=96a007e611eb4be3b9bd185be15e4112, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:33.504243 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001f2af-450b-41a5-b73c-c4c906d9d59f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:33.519173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1123/2260] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:33 2022] PUT /identity/v3/projects/7e699ac2f7b14177a7a41f3b9ea2beaa/users/96a007e611eb4be3b9bd185be15e4112/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:33.531403 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce59ae-6141-411a-bc7d-637e1ef8bbee None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:33.532033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce59ae-6141-411a-bc7d-637e1ef8bbee None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:33.532380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce59ae-6141-411a-bc7d-637e1ef8bbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:33.566900 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-35ce59ae-6141-411a-bc7d-637e1ef8bbee None None] MFA Rules not processed for user `96a007e611eb4be3b9bd185be15e4112`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:33.568396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-35ce59ae-6141-411a-bc7d-637e1ef8bbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:33.580432 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-feb4a3cc-bccc-4421-bd7c-598a50286e84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '389c21cefe094be18841369f0f1b8570'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:33.592565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-feb4a3cc-bccc-4421-bd7c-598a50286e84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '389c21cefe094be18841369f0f1b8570'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:33.594139 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-feb4a3cc-bccc-4421-bd7c-598a50286e84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '389c21cefe094be18841369f0f1b8570'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:33.637865 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-feb4a3cc-bccc-4421-bd7c-598a50286e84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '389c21cefe094be18841369f0f1b8570'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:33.691357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1138/2261] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:34:33 2022] DELETE /identity/v3/users/389c21cefe094be18841369f0f1b8570 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:34:33.762198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1124/2262] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:34:33.773137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4a3cc-bccc-4421-bd7c-598a50286e84 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:33.775431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-feb4a3cc-bccc-4421-bd7c-598a50286e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:33.989307 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4a3cc-bccc-4421-bd7c-598a50286e84 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:34.025906 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c571ea-ffb7-4563-9847-3f63b93091cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:34.033349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c571ea-ffb7-4563-9847-3f63b93091cf service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:34.036479 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c571ea-ffb7-4563-9847-3f63b93091cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:34.037040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c571ea-ffb7-4563-9847-3f63b93091cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:34.148696 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:34.264628 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c571ea-ffb7-4563-9847-3f63b93091cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:34.268555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c571ea-ffb7-4563-9847-3f63b93091cf service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:34.319675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1139/2263] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:34.567597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:34.671041 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4658808-ec78-4c86-8da4-0faf811e09bc None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:34.672015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4658808-ec78-4c86-8da4-0faf811e09bc None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:34.673289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4658808-ec78-4c86-8da4-0faf811e09bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:34.748180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-e4658808-ec78-4c86-8da4-0faf811e09bc None None] MFA Rules not processed for user `4e75a82cc7e94702ba0080fc65d6eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:34.750346 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e4658808-ec78-4c86-8da4-0faf811e09bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:34.982746 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1125/2264] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Jul 20 15:34:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:34:35.112988 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c571ea-ffb7-4563-9847-3f63b93091cf service cinder] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:35.117601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e5c571ea-ffb7-4563-9847-3f63b93091cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:35.183987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4658808-ec78-4c86-8da4-0faf811e09bc None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:35.186089 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e4658808-ec78-4c86-8da4-0faf811e09bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:35.294501 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c571ea-ffb7-4563-9847-3f63b93091cf service cinder] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:35.305619 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b6f9d-c550-44f8-bf71-74699d87e2fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0d719c0eb04a3b8a228c6ce70fdf98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:35.306339 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b6f9d-c550-44f8-bf71-74699d87e2fc service swift] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:35.306630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b6f9d-c550-44f8-bf71-74699d87e2fc service swift] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:35.306912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b6f9d-c550-44f8-bf71-74699d87e2fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:35.317807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b6f9d-c550-44f8-bf71-74699d87e2fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:35.319647 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b6f9d-c550-44f8-bf71-74699d87e2fc service swift] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:35.331581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1140/2265] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Jul 20 15:34:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:34:35.360341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4658808-ec78-4c86-8da4-0faf811e09bc None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:35.393281 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6158ea-2874-411d-8558-a22c16e3bc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:35.393281 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6158ea-2874-411d-8558-a22c16e3bc52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:35.393281 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6158ea-2874-411d-8558-a22c16e3bc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:35.401096 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6158ea-2874-411d-8558-a22c16e3bc52 admin admin] PATH_INFO: `/v3/projects/18b8bc855d7246b2a773de85f151ec63` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:35.422377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6158ea-2874-411d-8558-a22c16e3bc52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18b8bc855d7246b2a773de85f151ec63)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:35.422377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6158ea-2874-411d-8558-a22c16e3bc52 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:35.516975 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:35.592440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b6f9d-c550-44f8-bf71-74699d87e2fc service swift] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:35.604054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-5c0b6f9d-c550-44f8-bf71-74699d87e2fc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:35.626908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b6f9d-c550-44f8-bf71-74699d87e2fc service swift] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:35.632051 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f9d62-cfdd-40a8-979b-a494304ed3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:35.633088 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f9d62-cfdd-40a8-979b-a494304ed3cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:35.633480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f9d62-cfdd-40a8-979b-a494304ed3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:35.633867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f9d62-cfdd-40a8-979b-a494304ed3cc admin admin] PATH_INFO: `/v3/users/87bdbc4bad434d7ea9273466532ee7e1` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:35.659801 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f9d62-cfdd-40a8-979b-a494304ed3cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=87bdbc4bad434d7ea9273466532ee7e1)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:35.668856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f9d62-cfdd-40a8-979b-a494304ed3cc admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:35.696474 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-4e6158ea-2874-411d-8558-a22c16e3bc52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18b8bc855d7246b2a773de85f151ec63', 'request_id': 'req-4e6158ea-2874-411d-8558-a22c16e3bc52'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:35.723953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:35.802777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-4e6158ea-2874-411d-8558-a22c16e3bc52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18b8bc855d7246b2a773de85f151ec63', 'request_id': 'req-4e6158ea-2874-411d-8558-a22c16e3bc52'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:35.811463 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-4e6158ea-2874-411d-8558-a22c16e3bc52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18b8bc855d7246b2a773de85f151ec63', 'request_id': 'req-4e6158ea-2874-411d-8558-a22c16e3bc52'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:35.839870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-4e6158ea-2874-411d-8558-a22c16e3bc52 admin admin] The token cache is being invalidate because project 18b8bc855d7246b2a773de85f151ec63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:34:35.841961 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-4e6158ea-2874-411d-8558-a22c16e3bc52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:35.846227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-4e6158ea-2874-411d-8558-a22c16e3bc52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:35.849512 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1126/2266] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:34:35 2022] DELETE /identity/v3/projects/18b8bc855d7246b2a773de85f151ec63 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:35.854959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b471c-e94d-46e6-a9a2-48a41e551f05 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:35.855424 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b471c-e94d-46e6-a9a2-48a41e551f05 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:35.855897 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b471c-e94d-46e6-a9a2-48a41e551f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:35.905534 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-ed2b471c-e94d-46e6-a9a2-48a41e551f05 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:35.906855 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ed2b471c-e94d-46e6-a9a2-48a41e551f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:35.985419 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f10f9d62-cfdd-40a8-979b-a494304ed3cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87bdbc4bad434d7ea9273466532ee7e1'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:36.083194 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1127/2267] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:36.100664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f10f9d62-cfdd-40a8-979b-a494304ed3cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87bdbc4bad434d7ea9273466532ee7e1'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:36.129583 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f10f9d62-cfdd-40a8-979b-a494304ed3cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87bdbc4bad434d7ea9273466532ee7e1'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:36.139438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f10f9d62-cfdd-40a8-979b-a494304ed3cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87bdbc4bad434d7ea9273466532ee7e1'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:36.146349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ed77f-641b-4bff-8911-7d060f46cde5 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:36.146843 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ed77f-641b-4bff-8911-7d060f46cde5 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:36.147284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ed77f-641b-4bff-8911-7d060f46cde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:36.171261 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f10f9d62-cfdd-40a8-979b-a494304ed3cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87bdbc4bad434d7ea9273466532ee7e1'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:36.176893 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-4e8ed77f-641b-4bff-8911-7d060f46cde5 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:36.182404 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4e8ed77f-641b-4bff-8911-7d060f46cde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:36.190370 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1141/2268] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:34:35 2022] DELETE /identity/v3/users/87bdbc4bad434d7ea9273466532ee7e1 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:34:36.198437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1128/2269] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:34:36.205778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f9d62-cfdd-40a8-979b-a494304ed3cc admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:36.206864 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa44ecf-398a-4ea5-9fa3-b03f800ed6ba None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:36.207125 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa44ecf-398a-4ea5-9fa3-b03f800ed6ba None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:36.207403 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa44ecf-398a-4ea5-9fa3-b03f800ed6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:36.272256 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-2aa44ecf-398a-4ea5-9fa3-b03f800ed6ba None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:36.274237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-2aa44ecf-398a-4ea5-9fa3-b03f800ed6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:36.415124 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f9d62-cfdd-40a8-979b-a494304ed3cc admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:36.431962 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fdc1e-a1a2-44ad-b3b0-0311329c0889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:36.432662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fdc1e-a1a2-44ad-b3b0-0311329c0889 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:36.433105 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fdc1e-a1a2-44ad-b3b0-0311329c0889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:36.433396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fdc1e-a1a2-44ad-b3b0-0311329c0889 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:36.434726 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fdc1e-a1a2-44ad-b3b0-0311329c0889 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:34:36.435058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fdc1e-a1a2-44ad-b3b0-0311329c0889 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:36.436736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fdc1e-a1a2-44ad-b3b0-0311329c0889 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:36.474386 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1142/2270] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:34:36.478174 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fdc1e-a1a2-44ad-b3b0-0311329c0889 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:36.511396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fdc1e-a1a2-44ad-b3b0-0311329c0889 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:36.512817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1129/2271] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:36.525946 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa44ecf-398a-4ea5-9fa3-b03f800ed6ba None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:36.527720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-2aa44ecf-398a-4ea5-9fa3-b03f800ed6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:36.554677 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bd0cc-fc0f-4f03-9bcd-ca65eafa3e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:36.555671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bd0cc-fc0f-4f03-9bcd-ca65eafa3e23 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:36.556224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bd0cc-fc0f-4f03-9bcd-ca65eafa3e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:36.556665 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bd0cc-fc0f-4f03-9bcd-ca65eafa3e23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:36.557720 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0bd0cc-fc0f-4f03-9bcd-ca65eafa3e23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:36.566397 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0bd0cc-fc0f-4f03-9bcd-ca65eafa3e23 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:36.683304 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:36.725104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1143/2272] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:34:36 2022] POST /identity/v3/projects => generated 377 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:36.741307 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bd0cc-fc0f-4f03-9bcd-ca65eafa3e23 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:36.748130 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa44ecf-398a-4ea5-9fa3-b03f800ed6ba None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:36.828883 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e4d97-63d1-45ea-b56b-b3c3b1f96113 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:36.829988 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e4d97-63d1-45ea-b56b-b3c3b1f96113 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:36.830645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e4d97-63d1-45ea-b56b-b3c3b1f96113 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:36.830951 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e4d97-63d1-45ea-b56b-b3c3b1f96113 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:36.846972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bd0cc-fc0f-4f03-9bcd-ca65eafa3e23 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:36.865359 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51289a-2013-4519-b993-c064c51e0574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:36.866411 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51289a-2013-4519-b993-c064c51e0574 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:36.867178 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51289a-2013-4519-b993-c064c51e0574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:36.868799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51289a-2013-4519-b993-c064c51e0574 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:36.870523 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51289a-2013-4519-b993-c064c51e0574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:36.874120 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51289a-2013-4519-b993-c064c51e0574 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:36.941869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e4d97-63d1-45ea-b56b-b3c3b1f96113 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:36.943673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e4d97-63d1-45ea-b56b-b3c3b1f96113 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:37.029440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1130/2273] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:37.082365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1144/2274] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:34:36 2022] POST /identity/v3/users => generated 289 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:37.086632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e4d97-63d1-45ea-b56b-b3c3b1f96113 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:37.133429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e4d97-63d1-45ea-b56b-b3c3b1f96113 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:37.156817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13766ac-0b3e-4fbc-8005-98e07fc0e25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:37.157938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13766ac-0b3e-4fbc-8005-98e07fc0e25d admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:37.158539 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13766ac-0b3e-4fbc-8005-98e07fc0e25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:37.159166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13766ac-0b3e-4fbc-8005-98e07fc0e25d admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:37.160579 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13766ac-0b3e-4fbc-8005-98e07fc0e25d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:37.170247 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151d575-700f-4622-bf8d-828df6ff9c49 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:37.170889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151d575-700f-4622-bf8d-828df6ff9c49 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:37.170889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151d575-700f-4622-bf8d-828df6ff9c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:37.176922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13766ac-0b3e-4fbc-8005-98e07fc0e25d admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:37.225296 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-d151d575-700f-4622-bf8d-828df6ff9c49 None None] MFA Rules not processed for user `d8a92f26477b40dcaf77913e55c5fa6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:37.226401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1131/2275] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:37.226718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d151d575-700f-4622-bf8d-828df6ff9c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:37.227371 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13766ac-0b3e-4fbc-8005-98e07fc0e25d admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:37.256725 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13766ac-0b3e-4fbc-8005-98e07fc0e25d admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:37.276511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f287ad-1e82-447f-8955-22f82d08804f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:37.277203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f287ad-1e82-447f-8955-22f82d08804f admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:37.277501 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f287ad-1e82-447f-8955-22f82d08804f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:37.277815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f287ad-1e82-447f-8955-22f82d08804f admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:37.278782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f287ad-1e82-447f-8955-22f82d08804f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:37.284014 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f287ad-1e82-447f-8955-22f82d08804f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:37.315561 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1132/2276] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:37.319682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f287ad-1e82-447f-8955-22f82d08804f admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:37.322448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1145/2277] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:37.332153 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f287ad-1e82-447f-8955-22f82d08804f admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:37.344010 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb819cd-a97c-47e3-92ba-81bef517f37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:37.345021 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb819cd-a97c-47e3-92ba-81bef517f37a admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:37.348022 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb819cd-a97c-47e3-92ba-81bef517f37a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:37.356877 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb819cd-a97c-47e3-92ba-81bef517f37a admin admin] PATH_INFO: `/v3/projects/29c857195a8c464f9cc94a679e5ef214/users/9e03db243a974515b69ea065960d3030/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:37.374284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb819cd-a97c-47e3-92ba-81bef517f37a admin admin] RBAC: Authorizing `identity:create_grant(project_id=29c857195a8c464f9cc94a679e5ef214, user_id=9e03db243a974515b69ea065960d3030, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:37.377529 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb819cd-a97c-47e3-92ba-81bef517f37a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:37.452327 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:37.534472 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f390a-ae04-4450-9b39-2420d8e87687 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:37.534850 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1133/2278] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:37 2022] PUT /identity/v3/projects/29c857195a8c464f9cc94a679e5ef214/users/9e03db243a974515b69ea065960d3030/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:34:37.535200 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f390a-ae04-4450-9b39-2420d8e87687 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:37.535634 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f390a-ae04-4450-9b39-2420d8e87687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:37.592698 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6b6f390a-ae04-4450-9b39-2420d8e87687 None None] MFA Rules not processed for user `9e03db243a974515b69ea065960d3030`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:37.594725 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6b6f390a-ae04-4450-9b39-2420d8e87687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:37.731856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1146/2279] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:34:37.738609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb819cd-a97c-47e3-92ba-81bef517f37a admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:37.899444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb819cd-a97c-47e3-92ba-81bef517f37a admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:37.909410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4ef33-7fab-48b4-af1f-b4239b26cde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:37.914242 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4ef33-7fab-48b4-af1f-b4239b26cde1 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:37.915045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4ef33-7fab-48b4-af1f-b4239b26cde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:37.915726 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4ef33-7fab-48b4-af1f-b4239b26cde1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:37.917336 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4ef33-7fab-48b4-af1f-b4239b26cde1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:37.921739 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4ef33-7fab-48b4-af1f-b4239b26cde1 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:38.044976 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f390a-ae04-4450-9b39-2420d8e87687 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:38.048336 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6b6f390a-ae04-4450-9b39-2420d8e87687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:38.113054 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:38.142190 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1134/2280] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:34:37 2022] POST /identity/v3/projects => generated 377 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:38.144855 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4ef33-7fab-48b4-af1f-b4239b26cde1 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:38.161260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f390a-ae04-4450-9b39-2420d8e87687 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:38.308352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351b783-03b0-4f8b-b0d4-20df623050ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:38.311368 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351b783-03b0-4f8b-b0d4-20df623050ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:38.311813 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351b783-03b0-4f8b-b0d4-20df623050ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:38.312156 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351b783-03b0-4f8b-b0d4-20df623050ab admin admin] PATH_INFO: `/v3/projects/2139071aef014b79a1737bd9fadd3db9` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:38.313606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351b783-03b0-4f8b-b0d4-20df623050ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=2139071aef014b79a1737bd9fadd3db9)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:38.315943 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351b783-03b0-4f8b-b0d4-20df623050ab admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:38.385533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4ef33-7fab-48b4-af1f-b4239b26cde1 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:38.394431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c0c46-8519-48b7-afb7-da82b9075202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:38.395146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c0c46-8519-48b7-afb7-da82b9075202 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:38.395463 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c0c46-8519-48b7-afb7-da82b9075202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:38.395718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c0c46-8519-48b7-afb7-da82b9075202 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:38.396507 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c0c46-8519-48b7-afb7-da82b9075202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:38.398565 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c0c46-8519-48b7-afb7-da82b9075202 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:38.458128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0351b783-03b0-4f8b-b0d4-20df623050ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2139071aef014b79a1737bd9fadd3db9', 'request_id': 'req-0351b783-03b0-4f8b-b0d4-20df623050ab'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:38.522525 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:38.540446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1135/2281] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:34:38 2022] POST /identity/v3/users => generated 289 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:38.546008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c0c46-8519-48b7-afb7-da82b9075202 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:38.568033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c0c46-8519-48b7-afb7-da82b9075202 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:38.596723 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0351b783-03b0-4f8b-b0d4-20df623050ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2139071aef014b79a1737bd9fadd3db9', 'request_id': 'req-0351b783-03b0-4f8b-b0d4-20df623050ab'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:38.604336 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0351b783-03b0-4f8b-b0d4-20df623050ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2139071aef014b79a1737bd9fadd3db9', 'request_id': 'req-0351b783-03b0-4f8b-b0d4-20df623050ab'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:38.622229 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0351b783-03b0-4f8b-b0d4-20df623050ab admin admin] The token cache is being invalidate because project 2139071aef014b79a1737bd9fadd3db9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:34:38.622229 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0351b783-03b0-4f8b-b0d4-20df623050ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:38.622229 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0351b783-03b0-4f8b-b0d4-20df623050ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:38.633787 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1147/2282] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:34:38 2022] DELETE /identity/v3/projects/2139071aef014b79a1737bd9fadd3db9 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:38.634952 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351b783-03b0-4f8b-b0d4-20df623050ab admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:38.637578 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0351b783-03b0-4f8b-b0d4-20df623050ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:38.675810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:38.691899 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c656c-392c-4975-9f24-a7b6c9743b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:38.692706 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c656c-392c-4975-9f24-a7b6c9743b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:38.693061 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c656c-392c-4975-9f24-a7b6c9743b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:38.693414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c656c-392c-4975-9f24-a7b6c9743b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:38.694237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c656c-392c-4975-9f24-a7b6c9743b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:38.699126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c656c-392c-4975-9f24-a7b6c9743b08 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:38.720831 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1136/2283] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:34:38.735749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c656c-392c-4975-9f24-a7b6c9743b08 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:38.741542 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-519c656c-392c-4975-9f24-a7b6c9743b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:38.748033 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351b783-03b0-4f8b-b0d4-20df623050ab admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:38.756516 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c5622-ac33-48bd-9b5b-1aad7b3d3f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:38.757794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c5622-ac33-48bd-9b5b-1aad7b3d3f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:38.758174 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c5622-ac33-48bd-9b5b-1aad7b3d3f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:38.758638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c5622-ac33-48bd-9b5b-1aad7b3d3f39 admin admin] PATH_INFO: `/v3/users/96a007e611eb4be3b9bd185be15e4112` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:38.760104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c5622-ac33-48bd-9b5b-1aad7b3d3f39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96a007e611eb4be3b9bd185be15e4112)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:38.763933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c5622-ac33-48bd-9b5b-1aad7b3d3f39 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:38.869837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c656c-392c-4975-9f24-a7b6c9743b08 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:38.902310 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb653e5-565d-4603-b018-9ddd37726fd0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6c13596c2df438294bdac7c52ae8cb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:38.903713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb653e5-565d-4603-b018-9ddd37726fd0 service glance] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:38.904294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb653e5-565d-4603-b018-9ddd37726fd0 service glance] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:38.904687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb653e5-565d-4603-b018-9ddd37726fd0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:38.906982 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-9bb653e5-565d-4603-b018-9ddd37726fd0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:39.021596 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0f3c5622-ac33-48bd-9b5b-1aad7b3d3f39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96a007e611eb4be3b9bd185be15e4112'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:39.063323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb653e5-565d-4603-b018-9ddd37726fd0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:39.066067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb653e5-565d-4603-b018-9ddd37726fd0 service glance] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:39.087570 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1137/2284] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:39.087570 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb653e5-565d-4603-b018-9ddd37726fd0 service glance] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:39.087570 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-9bb653e5-565d-4603-b018-9ddd37726fd0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:39.127013 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb653e5-565d-4603-b018-9ddd37726fd0 service glance] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:39.135748 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888a3e7-4330-460d-b33c-2793528a9e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:39.137040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0f3c5622-ac33-48bd-9b5b-1aad7b3d3f39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96a007e611eb4be3b9bd185be15e4112'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:39.140792 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888a3e7-4330-460d-b33c-2793528a9e47 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:39.142105 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888a3e7-4330-460d-b33c-2793528a9e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:39.142439 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888a3e7-4330-460d-b33c-2793528a9e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:39.143260 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888a3e7-4330-460d-b33c-2793528a9e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:39.145242 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888a3e7-4330-460d-b33c-2793528a9e47 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:39.150034 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0f3c5622-ac33-48bd-9b5b-1aad7b3d3f39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96a007e611eb4be3b9bd185be15e4112'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:39.152044 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0f3c5622-ac33-48bd-9b5b-1aad7b3d3f39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96a007e611eb4be3b9bd185be15e4112'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:39.176458 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1138/2285] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:39.180853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888a3e7-4330-460d-b33c-2793528a9e47 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:39.190660 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a888a3e7-4330-460d-b33c-2793528a9e47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:39.210218 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0f3c5622-ac33-48bd-9b5b-1aad7b3d3f39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96a007e611eb4be3b9bd185be15e4112'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:39.228844 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1148/2286] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:34:38 2022] DELETE /identity/v3/users/96a007e611eb4be3b9bd185be15e4112 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:39.263666 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888a3e7-4330-460d-b33c-2793528a9e47 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:39.504903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af157ff-8dc6-4c45-8399-0442f637398f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:39.505842 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af157ff-8dc6-4c45-8399-0442f637398f admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:39.506127 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af157ff-8dc6-4c45-8399-0442f637398f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:39.506437 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af157ff-8dc6-4c45-8399-0442f637398f admin admin] PATH_INFO: `/v3/projects/8a16ede4a0c143c084285a796bbcd167/users/b0f2e61b180c4b74b2ce4829f5791109/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:39.557364 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af157ff-8dc6-4c45-8399-0442f637398f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a16ede4a0c143c084285a796bbcd167, user_id=b0f2e61b180c4b74b2ce4829f5791109, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:39.560480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af157ff-8dc6-4c45-8399-0442f637398f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:39.563888 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:39.569811 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16c73b-aa43-4b1a-a89e-35fd9134a6ed None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:39.570592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16c73b-aa43-4b1a-a89e-35fd9134a6ed None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:39.571968 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16c73b-aa43-4b1a-a89e-35fd9134a6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:39.602832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6d16c73b-aa43-4b1a-a89e-35fd9134a6ed None None] MFA Rules not processed for user `4e75a82cc7e94702ba0080fc65d6eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:39.604248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6d16c73b-aa43-4b1a-a89e-35fd9134a6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:39.709240 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1149/2287] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Jul 20 15:34:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:34:39.713717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1139/2288] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:39 2022] PUT /identity/v3/projects/8a16ede4a0c143c084285a796bbcd167/users/b0f2e61b180c4b74b2ce4829f5791109/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:39.725180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1d4bf-3e4b-4fb4-b22e-752913f44cf6 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:39.726507 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1d4bf-3e4b-4fb4-b22e-752913f44cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:39.727452 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1d4bf-3e4b-4fb4-b22e-752913f44cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:39.733825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af157ff-8dc6-4c45-8399-0442f637398f admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:39.736060 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-3af157ff-8dc6-4c45-8399-0442f637398f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:39.784269 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:39.805394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-e6c1d4bf-3e4b-4fb4-b22e-752913f44cf6 None None] MFA Rules not processed for user `b0f2e61b180c4b74b2ce4829f5791109`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:39.806462 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:39.806594 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e6c1d4bf-3e4b-4fb4-b22e-752913f44cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:39.858880 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af157ff-8dc6-4c45-8399-0442f637398f admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:39.871044 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab801db8-45af-4a39-bc46-adaa7f84b83d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0d719c0eb04a3b8a228c6ce70fdf98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:39.874724 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab801db8-45af-4a39-bc46-adaa7f84b83d service swift] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:39.878440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab801db8-45af-4a39-bc46-adaa7f84b83d service swift] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:39.878440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab801db8-45af-4a39-bc46-adaa7f84b83d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:39.975742 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1150/2289] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:39.991609 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1d4bf-3e4b-4fb4-b22e-752913f44cf6 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:40.004773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab801db8-45af-4a39-bc46-adaa7f84b83d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:40.009470 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab801db8-45af-4a39-bc46-adaa7f84b83d service swift] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:40.033716 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1140/2290] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Jul 20 15:34:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 15:34:40.111916 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab801db8-45af-4a39-bc46-adaa7f84b83d service swift] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:40.113517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ab801db8-45af-4a39-bc46-adaa7f84b83d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:40.132037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1d4bf-3e4b-4fb4-b22e-752913f44cf6 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:40.138542 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191b2bd-1305-424f-b0f0-227f482ad4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:40.139683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191b2bd-1305-424f-b0f0-227f482ad4a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:40.139683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191b2bd-1305-424f-b0f0-227f482ad4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:40.140601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191b2bd-1305-424f-b0f0-227f482ad4a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:40.140601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191b2bd-1305-424f-b0f0-227f482ad4a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:40.145317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191b2bd-1305-424f-b0f0-227f482ad4a8 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:40.153956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab801db8-45af-4a39-bc46-adaa7f84b83d service swift] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:40.174865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec451ee1-e642-486a-9942-b6889bf18cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:40.176004 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec451ee1-e642-486a-9942-b6889bf18cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:40.176627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec451ee1-e642-486a-9942-b6889bf18cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:40.176983 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec451ee1-e642-486a-9942-b6889bf18cc2 admin admin] PATH_INFO: `/v3/projects/7e699ac2f7b14177a7a41f3b9ea2beaa` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:40.181943 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec451ee1-e642-486a-9942-b6889bf18cc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e699ac2f7b14177a7a41f3b9ea2beaa)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:40.195666 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec451ee1-e642-486a-9942-b6889bf18cc2 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:40.296369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1151/2291] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:34:39 2022] POST /identity/v3/projects => generated 375 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:40.299292 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191b2bd-1305-424f-b0f0-227f482ad4a8 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:40.416414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:40.451349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ec451ee1-e642-486a-9942-b6889bf18cc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e699ac2f7b14177a7a41f3b9ea2beaa', 'request_id': 'req-ec451ee1-e642-486a-9942-b6889bf18cc2'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:40.511236 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191b2bd-1305-424f-b0f0-227f482ad4a8 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:40.576386 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ec451ee1-e642-486a-9942-b6889bf18cc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e699ac2f7b14177a7a41f3b9ea2beaa', 'request_id': 'req-ec451ee1-e642-486a-9942-b6889bf18cc2'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:40.585602 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ec451ee1-e642-486a-9942-b6889bf18cc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e699ac2f7b14177a7a41f3b9ea2beaa', 'request_id': 'req-ec451ee1-e642-486a-9942-b6889bf18cc2'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:40.637598 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546290e-a61f-4f46-b05c-4fc29f8fd149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:40.638443 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546290e-a61f-4f46-b05c-4fc29f8fd149 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:40.639375 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546290e-a61f-4f46-b05c-4fc29f8fd149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:40.639915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ec451ee1-e642-486a-9942-b6889bf18cc2 admin admin] The token cache is being invalidate because project 7e699ac2f7b14177a7a41f3b9ea2beaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:34:40.640228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ec451ee1-e642-486a-9942-b6889bf18cc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:40.640940 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ec451ee1-e642-486a-9942-b6889bf18cc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:40.641750 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546290e-a61f-4f46-b05c-4fc29f8fd149 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:40.642550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546290e-a61f-4f46-b05c-4fc29f8fd149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:40.643776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1141/2292] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:34:40 2022] DELETE /identity/v3/projects/7e699ac2f7b14177a7a41f3b9ea2beaa => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:34:40.644570 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546290e-a61f-4f46-b05c-4fc29f8fd149 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:40.647880 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec451ee1-e642-486a-9942-b6889bf18cc2 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:40.664646 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ec451ee1-e642-486a-9942-b6889bf18cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:40.697407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec451ee1-e642-486a-9942-b6889bf18cc2 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:40.729219 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cee789-74f3-4175-bf6c-552b33ea465d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:40.730156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cee789-74f3-4175-bf6c-552b33ea465d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:40.730564 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cee789-74f3-4175-bf6c-552b33ea465d admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:40.731379 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cee789-74f3-4175-bf6c-552b33ea465d admin admin] PATH_INFO: `/v3/users/ed454faac2814963a6e89c848bdb2dbd` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:40.736996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cee789-74f3-4175-bf6c-552b33ea465d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed454faac2814963a6e89c848bdb2dbd)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:40.739433 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cee789-74f3-4175-bf6c-552b33ea465d admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:40.763402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1152/2293] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:34:40 2022] POST /identity/v3/users => generated 294 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:40.766179 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407cf75-b128-4034-b813-c196a750fedf None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:40.767288 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407cf75-b128-4034-b813-c196a750fedf None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:40.767571 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407cf75-b128-4034-b813-c196a750fedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:40.823380 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-1407cf75-b128-4034-b813-c196a750fedf None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:40.828011 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1407cf75-b128-4034-b813-c196a750fedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:40.852864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1153/2294] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:40.854976 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407cf75-b128-4034-b813-c196a750fedf None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:40.855381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1407cf75-b128-4034-b813-c196a750fedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:40.892774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407cf75-b128-4034-b813-c196a750fedf None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:40.907052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7282414f-0cb0-49bb-9497-eafe29a14f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:40.907740 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7282414f-0cb0-49bb-9497-eafe29a14f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:40.908025 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7282414f-0cb0-49bb-9497-eafe29a14f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:40.909545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7282414f-0cb0-49bb-9497-eafe29a14f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:40.911850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7282414f-0cb0-49bb-9497-eafe29a14f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:40.914588 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7282414f-0cb0-49bb-9497-eafe29a14f58 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:40.950687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1154/2295] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:40.953440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073aa4b-13a1-4e63-b630-19b8b5a86308 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:40.954149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073aa4b-13a1-4e63-b630-19b8b5a86308 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:40.954599 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073aa4b-13a1-4e63-b630-19b8b5a86308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:40.974772 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:40.986809 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6073aa4b-13a1-4e63-b630-19b8b5a86308 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:40.988338 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6073aa4b-13a1-4e63-b630-19b8b5a86308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:40.998091 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1155/2296] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:40.999255 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073aa4b-13a1-4e63-b630-19b8b5a86308 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:41.016809 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073aa4b-13a1-4e63-b630-19b8b5a86308 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:41.020228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-85cee789-74f3-4175-bf6c-552b33ea465d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed454faac2814963a6e89c848bdb2dbd'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:41.022568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59040f8d-11a4-4463-862d-16435d097ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:41.023723 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59040f8d-11a4-4463-862d-16435d097ef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:41.024014 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59040f8d-11a4-4463-862d-16435d097ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:41.024295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59040f8d-11a4-4463-862d-16435d097ef7 admin admin] PATH_INFO: `/v3/projects/6ae5869ac8df490e9489dce20e5ca9cf/users/21230dc4e66e45fab10ea8302d773db6/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:41.060460 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59040f8d-11a4-4463-862d-16435d097ef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae5869ac8df490e9489dce20e5ca9cf, user_id=21230dc4e66e45fab10ea8302d773db6, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:41.063621 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59040f8d-11a4-4463-862d-16435d097ef7 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:41.168556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-85cee789-74f3-4175-bf6c-552b33ea465d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed454faac2814963a6e89c848bdb2dbd'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:41.169285 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-85cee789-74f3-4175-bf6c-552b33ea465d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed454faac2814963a6e89c848bdb2dbd'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:41.181053 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1156/2297] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:40 2022] PUT /identity/v3/projects/6ae5869ac8df490e9489dce20e5ca9cf/users/21230dc4e66e45fab10ea8302d773db6/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:41.183167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59040f8d-11a4-4463-862d-16435d097ef7 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:41.189095 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-59040f8d-11a4-4463-862d-16435d097ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:41.217699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-85cee789-74f3-4175-bf6c-552b33ea465d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed454faac2814963a6e89c848bdb2dbd'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:41.279957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-85cee789-74f3-4175-bf6c-552b33ea465d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed454faac2814963a6e89c848bdb2dbd'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:41.329416 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1142/2298] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:34:40 2022] DELETE /identity/v3/users/ed454faac2814963a6e89c848bdb2dbd => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:41.341583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cee789-74f3-4175-bf6c-552b33ea465d admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:41.342125 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-85cee789-74f3-4175-bf6c-552b33ea465d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:41.394723 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59040f8d-11a4-4463-862d-16435d097ef7 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:41.420834 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:41.463535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cee789-74f3-4175-bf6c-552b33ea465d admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:41.480649 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0731f-d618-496a-9574-f7a403e87c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:41.480649 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0731f-d618-496a-9574-f7a403e87c8e admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:41.480649 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0731f-d618-496a-9574-f7a403e87c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:41.480649 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0731f-d618-496a-9574-f7a403e87c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:41.481729 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced0731f-d618-496a-9574-f7a403e87c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:41.483512 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced0731f-d618-496a-9574-f7a403e87c8e admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:41.506606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1143/2299] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:41.511202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564c894-c177-40d9-942f-eaff350d3288 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:41.511494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564c894-c177-40d9-942f-eaff350d3288 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:41.511882 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564c894-c177-40d9-942f-eaff350d3288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:41.515153 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea37f3-5c6b-4523-9e2d-764abee30f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:41.516118 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea37f3-5c6b-4523-9e2d-764abee30f8e admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:41.516658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea37f3-5c6b-4523-9e2d-764abee30f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:41.517280 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea37f3-5c6b-4523-9e2d-764abee30f8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:41.518887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea37f3-5c6b-4523-9e2d-764abee30f8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:34:41.519389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea37f3-5c6b-4523-9e2d-764abee30f8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:41.530649 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea37f3-5c6b-4523-9e2d-764abee30f8e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:41.537208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1157/2300] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:34:41.541689 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea37f3-5c6b-4523-9e2d-764abee30f8e admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:41.542246 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-6564c894-c177-40d9-942f-eaff350d3288 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:41.546726 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-6564c894-c177-40d9-942f-eaff350d3288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:41.556504 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1144/2301] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:41.559519 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea37f3-5c6b-4523-9e2d-764abee30f8e admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:41.560249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564c894-c177-40d9-942f-eaff350d3288 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:41.570902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92262736-b136-41bc-9745-cce6694f8a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:41.572303 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92262736-b136-41bc-9745-cce6694f8a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:41.572794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92262736-b136-41bc-9745-cce6694f8a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:41.573093 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564c894-c177-40d9-942f-eaff350d3288 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:41.573508 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92262736-b136-41bc-9745-cce6694f8a00 admin admin] PATH_INFO: `/v3/projects/6ae5869ac8df490e9489dce20e5ca9cf/users/21230dc4e66e45fab10ea8302d773db6/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:41.577441 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92262736-b136-41bc-9745-cce6694f8a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae5869ac8df490e9489dce20e5ca9cf, user_id=21230dc4e66e45fab10ea8302d773db6, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:41.578987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b925c5-b5ba-494f-86fb-38f6c6e11d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:41.579753 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b925c5-b5ba-494f-86fb-38f6c6e11d2f admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:41.580110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b925c5-b5ba-494f-86fb-38f6c6e11d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:41.580454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b925c5-b5ba-494f-86fb-38f6c6e11d2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:41.580752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92262736-b136-41bc-9745-cce6694f8a00 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:41.581327 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b925c5-b5ba-494f-86fb-38f6c6e11d2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:41.583641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b925c5-b5ba-494f-86fb-38f6c6e11d2f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:41.599089 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1158/2302] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:41 2022] PUT /identity/v3/projects/6ae5869ac8df490e9489dce20e5ca9cf/users/21230dc4e66e45fab10ea8302d773db6/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:41.600104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92262736-b136-41bc-9745-cce6694f8a00 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:41.602150 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-92262736-b136-41bc-9745-cce6694f8a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:41.705850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92262736-b136-41bc-9745-cce6694f8a00 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:41.710148 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0477e9f-07d0-43f1-b40b-b302ed6a04b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:41.710962 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0477e9f-07d0-43f1-b40b-b302ed6a04b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:41.711350 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0477e9f-07d0-43f1-b40b-b302ed6a04b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:41.711745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0477e9f-07d0-43f1-b40b-b302ed6a04b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:41.811654 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0477e9f-07d0-43f1-b40b-b302ed6a04b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:41.815275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0477e9f-07d0-43f1-b40b-b302ed6a04b6 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:41.830571 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1159/2303] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:41.834220 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78a29c-3f7a-4cdd-a9a8-7961fd131a93 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:41.834220 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78a29c-3f7a-4cdd-a9a8-7961fd131a93 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:41.834220 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78a29c-3f7a-4cdd-a9a8-7961fd131a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:41.898046 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1145/2304] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:34:41 2022] POST /identity/v3/projects => generated 361 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:41.902217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b925c5-b5ba-494f-86fb-38f6c6e11d2f admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:41.913786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-3f78a29c-3f7a-4cdd-a9a8-7961fd131a93 None None] MFA Rules not processed for user `21230dc4e66e45fab10ea8302d773db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:41.916077 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-3f78a29c-3f7a-4cdd-a9a8-7961fd131a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:41.987138 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:42.056099 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b925c5-b5ba-494f-86fb-38f6c6e11d2f admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:42.066683 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dd709-6cac-4bc5-9607-730ef0143da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:42.067755 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dd709-6cac-4bc5-9607-730ef0143da0 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:42.068191 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dd709-6cac-4bc5-9607-730ef0143da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:42.072042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dd709-6cac-4bc5-9607-730ef0143da0 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:42.073123 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785dd709-6cac-4bc5-9607-730ef0143da0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:42.076050 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785dd709-6cac-4bc5-9607-730ef0143da0 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:42.103618 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1160/2305] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:42.115204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78a29c-3f7a-4cdd-a9a8-7961fd131a93 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:42.122217 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-3f78a29c-3f7a-4cdd-a9a8-7961fd131a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:42.196832 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1146/2306] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:34:41 2022] POST /identity/v3/users => generated 281 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:42.201910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dd709-6cac-4bc5-9607-730ef0143da0 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:42.227756 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dd709-6cac-4bc5-9607-730ef0143da0 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:42.228444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78a29c-3f7a-4cdd-a9a8-7961fd131a93 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:42.235301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4f022-6e76-4b86-9b88-e169bcc87cc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:42.239316 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4f022-6e76-4b86-9b88-e169bcc87cc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:42.239731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4f022-6e76-4b86-9b88-e169bcc87cc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:42.240119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4f022-6e76-4b86-9b88-e169bcc87cc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:42.244529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b4f4f022-6e76-4b86-9b88-e169bcc87cc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:42.246864 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e94de-532a-41f8-86c5-d3683f4a6cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:42.247561 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e94de-532a-41f8-86c5-d3683f4a6cad admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:42.248096 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e94de-532a-41f8-86c5-d3683f4a6cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:42.248381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e94de-532a-41f8-86c5-d3683f4a6cad admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:42.249345 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e94de-532a-41f8-86c5-d3683f4a6cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:42.250880 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e94de-532a-41f8-86c5-d3683f4a6cad admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:42.263585 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:42.275902 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1147/2307] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:34:42.278399 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e94de-532a-41f8-86c5-d3683f4a6cad admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:42.294636 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e94de-532a-41f8-86c5-d3683f4a6cad admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:42.303165 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b36d08-555a-4a98-91a1-b6c083a76afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:42.303945 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b36d08-555a-4a98-91a1-b6c083a76afb admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:42.304244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b36d08-555a-4a98-91a1-b6c083a76afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:42.304520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b36d08-555a-4a98-91a1-b6c083a76afb admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:42.305371 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b36d08-555a-4a98-91a1-b6c083a76afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:42.307094 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b36d08-555a-4a98-91a1-b6c083a76afb admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:42.328017 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1148/2308] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:42.328958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b36d08-555a-4a98-91a1-b6c083a76afb admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:42.354549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b36d08-555a-4a98-91a1-b6c083a76afb admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:42.376608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afaed8-e0f5-4d2f-89c7-e1153d01d655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:42.379196 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afaed8-e0f5-4d2f-89c7-e1153d01d655 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:42.386930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afaed8-e0f5-4d2f-89c7-e1153d01d655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:42.387230 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afaed8-e0f5-4d2f-89c7-e1153d01d655 admin admin] PATH_INFO: `/v3/projects/ffb685d7e14c40c8892a70656215ac90/users/6ec78a65777c47b89cb7dec707b9829c/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:42.419586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afaed8-e0f5-4d2f-89c7-e1153d01d655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb685d7e14c40c8892a70656215ac90, user_id=6ec78a65777c47b89cb7dec707b9829c, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:42.422741 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afaed8-e0f5-4d2f-89c7-e1153d01d655 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:42.423119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4f022-6e76-4b86-9b88-e169bcc87cc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:42.424836 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4f022-6e76-4b86-9b88-e169bcc87cc2 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:42.449630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1161/2309] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:34:42.535026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1149/2310] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:42 2022] PUT /identity/v3/projects/ffb685d7e14c40c8892a70656215ac90/users/6ec78a65777c47b89cb7dec707b9829c/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:42.547981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afce2ec-835e-4242-b47b-b21e9fcd5f11 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:42.548277 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afce2ec-835e-4242-b47b-b21e9fcd5f11 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:42.548584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afce2ec-835e-4242-b47b-b21e9fcd5f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:42.633918 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6afce2ec-835e-4242-b47b-b21e9fcd5f11 None None] MFA Rules not processed for user `6ec78a65777c47b89cb7dec707b9829c`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:42.638589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6afce2ec-835e-4242-b47b-b21e9fcd5f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:42.764315 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afaed8-e0f5-4d2f-89c7-e1153d01d655 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:42.765987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-43afaed8-e0f5-4d2f-89c7-e1153d01d655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:42.830889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1162/2311] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:42.879990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afce2ec-835e-4242-b47b-b21e9fcd5f11 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:42.881601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6afce2ec-835e-4242-b47b-b21e9fcd5f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:42.925712 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:43.026373 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afce2ec-835e-4242-b47b-b21e9fcd5f11 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:43.032999 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afaed8-e0f5-4d2f-89c7-e1153d01d655 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:43.038396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813e449-ae87-4fa5-b659-006030539271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:43.038851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952381b-66ee-48f3-aaa4-fa38eb5a2a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:43.039346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813e449-ae87-4fa5-b659-006030539271 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:43.039782 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813e449-ae87-4fa5-b659-006030539271 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:43.040159 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813e449-ae87-4fa5-b659-006030539271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:43.040641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952381b-66ee-48f3-aaa4-fa38eb5a2a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:43.041197 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952381b-66ee-48f3-aaa4-fa38eb5a2a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:43.041748 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952381b-66ee-48f3-aaa4-fa38eb5a2a8e admin admin] PATH_INFO: `/v3/projects/9adb6a1378e04df5978660014d2c44f7` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:43.043842 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952381b-66ee-48f3-aaa4-fa38eb5a2a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9adb6a1378e04df5978660014d2c44f7)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:43.048507 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952381b-66ee-48f3-aaa4-fa38eb5a2a8e admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:43.062243 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813e449-ae87-4fa5-b659-006030539271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:43.064191 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813e449-ae87-4fa5-b659-006030539271 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:43.088728 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1163/2312] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:43.110165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813e449-ae87-4fa5-b659-006030539271 service cinder] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:43.139681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813e449-ae87-4fa5-b659-006030539271 service cinder] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:43.156024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684aa0b3-8ace-4cb0-8527-a84714e6d600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:43.156926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684aa0b3-8ace-4cb0-8527-a84714e6d600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:43.157333 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684aa0b3-8ace-4cb0-8527-a84714e6d600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:43.157616 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684aa0b3-8ace-4cb0-8527-a84714e6d600 admin admin] PATH_INFO: `/v3/users/6ec78a65777c47b89cb7dec707b9829c` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:43.158914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684aa0b3-8ace-4cb0-8527-a84714e6d600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ec78a65777c47b89cb7dec707b9829c)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:43.160877 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684aa0b3-8ace-4cb0-8527-a84714e6d600 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:43.258316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-7952381b-66ee-48f3-aaa4-fa38eb5a2a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9adb6a1378e04df5978660014d2c44f7', 'request_id': 'req-7952381b-66ee-48f3-aaa4-fa38eb5a2a8e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:43.350315 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-7952381b-66ee-48f3-aaa4-fa38eb5a2a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9adb6a1378e04df5978660014d2c44f7', 'request_id': 'req-7952381b-66ee-48f3-aaa4-fa38eb5a2a8e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:43.367161 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-7952381b-66ee-48f3-aaa4-fa38eb5a2a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9adb6a1378e04df5978660014d2c44f7', 'request_id': 'req-7952381b-66ee-48f3-aaa4-fa38eb5a2a8e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:43.392778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-684aa0b3-8ace-4cb0-8527-a84714e6d600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ec78a65777c47b89cb7dec707b9829c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:43.400634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-7952381b-66ee-48f3-aaa4-fa38eb5a2a8e admin admin] The token cache is being invalidate because project 9adb6a1378e04df5978660014d2c44f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:34:43.411975 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-7952381b-66ee-48f3-aaa4-fa38eb5a2a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:43.414891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-7952381b-66ee-48f3-aaa4-fa38eb5a2a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:43.424636 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1150/2313] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:34:42 2022] DELETE /identity/v3/projects/9adb6a1378e04df5978660014d2c44f7 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:34:43.426183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691a0e3-9c08-4170-8735-e80a54dc9e72 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:43.426594 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691a0e3-9c08-4170-8735-e80a54dc9e72 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:43.427010 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691a0e3-9c08-4170-8735-e80a54dc9e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:43.456970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-4691a0e3-9c08-4170-8735-e80a54dc9e72 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:43.459320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4691a0e3-9c08-4170-8735-e80a54dc9e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:43.530209 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-684aa0b3-8ace-4cb0-8527-a84714e6d600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ec78a65777c47b89cb7dec707b9829c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:43.546052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-684aa0b3-8ace-4cb0-8527-a84714e6d600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ec78a65777c47b89cb7dec707b9829c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:43.547907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-684aa0b3-8ace-4cb0-8527-a84714e6d600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ec78a65777c47b89cb7dec707b9829c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:43.560560 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:43.577369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-684aa0b3-8ace-4cb0-8527-a84714e6d600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ec78a65777c47b89cb7dec707b9829c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:43.584036 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1151/2314] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:43.586952 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1164/2315] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:34:43 2022] DELETE /identity/v3/users/6ec78a65777c47b89cb7dec707b9829c => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:43.592684 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df9837-b989-49cc-90ed-c94bcb69f8b4 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:43.595195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df9837-b989-49cc-90ed-c94bcb69f8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:43.595195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df9837-b989-49cc-90ed-c94bcb69f8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:43.595469 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78aa047-df88-43db-b94d-15b3be420d2c None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:43.596089 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78aa047-df88-43db-b94d-15b3be420d2c None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:43.596628 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78aa047-df88-43db-b94d-15b3be420d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:43.626386 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-d8df9837-b989-49cc-90ed-c94bcb69f8b4 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:43.629809 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d8df9837-b989-49cc-90ed-c94bcb69f8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:43.633528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-c78aa047-df88-43db-b94d-15b3be420d2c None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:43.634879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c78aa047-df88-43db-b94d-15b3be420d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:43.810119 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1152/2316] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:43.814702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1165/2317] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:43.817452 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df9837-b989-49cc-90ed-c94bcb69f8b4 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:43.820553 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78aa047-df88-43db-b94d-15b3be420d2c None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:43.825861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c78aa047-df88-43db-b94d-15b3be420d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:43.838287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df9837-b989-49cc-90ed-c94bcb69f8b4 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:43.855387 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975737e-249c-4adb-8e70-e94f2edb184b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:43.855387 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975737e-249c-4adb-8e70-e94f2edb184b admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:43.855387 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975737e-249c-4adb-8e70-e94f2edb184b admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:43.855387 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975737e-249c-4adb-8e70-e94f2edb184b admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:43.856741 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975737e-249c-4adb-8e70-e94f2edb184b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:34:43.856741 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975737e-249c-4adb-8e70-e94f2edb184b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:43.863450 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975737e-249c-4adb-8e70-e94f2edb184b admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:43.899655 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1153/2318] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:34:43.913279 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975737e-249c-4adb-8e70-e94f2edb184b admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:43.917060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:43.938029 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975737e-249c-4adb-8e70-e94f2edb184b admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:43.949049 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54ead3-95d8-4f2c-9b3c-30f6f9bdeebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:43.950956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54ead3-95d8-4f2c-9b3c-30f6f9bdeebb admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:43.951411 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54ead3-95d8-4f2c-9b3c-30f6f9bdeebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:43.951870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54ead3-95d8-4f2c-9b3c-30f6f9bdeebb admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:43.952749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e54ead3-95d8-4f2c-9b3c-30f6f9bdeebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:43.958013 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e54ead3-95d8-4f2c-9b3c-30f6f9bdeebb admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:43.981708 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78aa047-df88-43db-b94d-15b3be420d2c None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:43.998051 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789a063-8bc7-4411-b314-3523a07ed734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:43.998974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789a063-8bc7-4411-b314-3523a07ed734 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:43.999348 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789a063-8bc7-4411-b314-3523a07ed734 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:43.999686 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789a063-8bc7-4411-b314-3523a07ed734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:44.025489 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e789a063-8bc7-4411-b314-3523a07ed734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:44.027180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e789a063-8bc7-4411-b314-3523a07ed734 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:44.070679 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1166/2319] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:44.138073 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1154/2320] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:34:43 2022] POST /identity/v3/projects => generated 375 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:44.138723 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789a063-8bc7-4411-b314-3523a07ed734 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:44.155555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789a063-8bc7-4411-b314-3523a07ed734 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:44.375326 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de1699-a278-4088-84f6-9c0104053f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:44.392262 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de1699-a278-4088-84f6-9c0104053f46 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:44.392262 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de1699-a278-4088-84f6-9c0104053f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:44.392262 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de1699-a278-4088-84f6-9c0104053f46 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:44.392262 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de1699-a278-4088-84f6-9c0104053f46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:44.392262 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de1699-a278-4088-84f6-9c0104053f46 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:44.656095 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1167/2321] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:34:44 2022] POST /identity/v3/users => generated 288 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:34:44.660266 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54ead3-95d8-4f2c-9b3c-30f6f9bdeebb admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:44.684884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54ead3-95d8-4f2c-9b3c-30f6f9bdeebb admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:44.749571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c43ad-df7a-4508-92a9-256710f6b11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:44.752150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c43ad-df7a-4508-92a9-256710f6b11d admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:44.752800 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c43ad-df7a-4508-92a9-256710f6b11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:44.753553 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c43ad-df7a-4508-92a9-256710f6b11d admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:44.759578 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c43ad-df7a-4508-92a9-256710f6b11d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:44.763033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c43ad-df7a-4508-92a9-256710f6b11d admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:44.799786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1155/2322] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:44.805763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de1699-a278-4088-84f6-9c0104053f46 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:44.837712 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de1699-a278-4088-84f6-9c0104053f46 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:44.861850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369ae7a-1a0d-4d89-aa81-c51f9b2b29b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:44.863906 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369ae7a-1a0d-4d89-aa81-c51f9b2b29b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:44.864343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369ae7a-1a0d-4d89-aa81-c51f9b2b29b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:44.864671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369ae7a-1a0d-4d89-aa81-c51f9b2b29b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:44.868493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369ae7a-1a0d-4d89-aa81-c51f9b2b29b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:44.874594 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369ae7a-1a0d-4d89-aa81-c51f9b2b29b5 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:44.900702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1168/2323] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:44.901790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c43ad-df7a-4508-92a9-256710f6b11d admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:44.916943 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c43ad-df7a-4508-92a9-256710f6b11d admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:44.936620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6940ab8-6147-4971-a1d0-83ee11b0ac77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:44.937711 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6940ab8-6147-4971-a1d0-83ee11b0ac77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:44.938106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6940ab8-6147-4971-a1d0-83ee11b0ac77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:44.938518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6940ab8-6147-4971-a1d0-83ee11b0ac77 admin admin] PATH_INFO: `/v3/projects/aa42974b2b0342fca28da5a3e98bba41/users/4f837f8db5504da08424e1ec2ed94f6f/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:44.964193 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6940ab8-6147-4971-a1d0-83ee11b0ac77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa42974b2b0342fca28da5a3e98bba41, user_id=4f837f8db5504da08424e1ec2ed94f6f, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:44.967685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369ae7a-1a0d-4d89-aa81-c51f9b2b29b5 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:44.968454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6940ab8-6147-4971-a1d0-83ee11b0ac77 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:44.973141 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f369ae7a-1a0d-4d89-aa81-c51f9b2b29b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:45.163452 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369ae7a-1a0d-4d89-aa81-c51f9b2b29b5 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:45.167875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016561c2-6992-40a6-bab5-1765c6eed36c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:45.169052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016561c2-6992-40a6-bab5-1765c6eed36c service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:45.169550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016561c2-6992-40a6-bab5-1765c6eed36c service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:45.169990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016561c2-6992-40a6-bab5-1765c6eed36c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:45.172104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-016561c2-6992-40a6-bab5-1765c6eed36c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:45.254767 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016561c2-6992-40a6-bab5-1765c6eed36c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:45.256699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016561c2-6992-40a6-bab5-1765c6eed36c service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:45.268316 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1169/2324] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:34:45.501283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1156/2325] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:44 2022] PUT /identity/v3/projects/aa42974b2b0342fca28da5a3e98bba41/users/4f837f8db5504da08424e1ec2ed94f6f/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:45.515072 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbb4c8-0486-4ceb-9ceb-24347cbe406b None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:45.515307 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbb4c8-0486-4ceb-9ceb-24347cbe406b None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:45.515733 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbb4c8-0486-4ceb-9ceb-24347cbe406b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:45.555523 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:45.559753 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-05dbb4c8-0486-4ceb-9ceb-24347cbe406b None None] MFA Rules not processed for user `4f837f8db5504da08424e1ec2ed94f6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:45.561305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-05dbb4c8-0486-4ceb-9ceb-24347cbe406b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:45.688135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1170/2326] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:45.700700 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6940ab8-6147-4971-a1d0-83ee11b0ac77 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:45.702776 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b6940ab8-6147-4971-a1d0-83ee11b0ac77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:45.784154 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6940ab8-6147-4971-a1d0-83ee11b0ac77 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:45.790973 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8ccd0-b040-4789-8e6a-894b5e8fb4d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:45.791656 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8ccd0-b040-4789-8e6a-894b5e8fb4d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:45.791952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8ccd0-b040-4789-8e6a-894b5e8fb4d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:45.792305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8ccd0-b040-4789-8e6a-894b5e8fb4d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:45.807374 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8ccd0-b040-4789-8e6a-894b5e8fb4d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:45.808560 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8ccd0-b040-4789-8e6a-894b5e8fb4d7 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:45.820844 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1157/2327] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:45.848930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbb4c8-0486-4ceb-9ceb-24347cbe406b None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:45.850982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-05dbb4c8-0486-4ceb-9ceb-24347cbe406b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:45.973105 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbb4c8-0486-4ceb-9ceb-24347cbe406b None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:45.993574 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619bf918-b18c-4068-88ff-5172acb5d617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:45.994610 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bf918-b18c-4068-88ff-5172acb5d617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:45.994852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bf918-b18c-4068-88ff-5172acb5d617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:45.997540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bf918-b18c-4068-88ff-5172acb5d617 admin admin] PATH_INFO: `/v3/projects/ffb685d7e14c40c8892a70656215ac90` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:45.999016 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619bf918-b18c-4068-88ff-5172acb5d617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffb685d7e14c40c8892a70656215ac90)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:46.002234 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619bf918-b18c-4068-88ff-5172acb5d617 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:46.223555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-619bf918-b18c-4068-88ff-5172acb5d617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffb685d7e14c40c8892a70656215ac90', 'request_id': 'req-619bf918-b18c-4068-88ff-5172acb5d617'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:46.349902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-619bf918-b18c-4068-88ff-5172acb5d617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffb685d7e14c40c8892a70656215ac90', 'request_id': 'req-619bf918-b18c-4068-88ff-5172acb5d617'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:46.361987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-619bf918-b18c-4068-88ff-5172acb5d617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffb685d7e14c40c8892a70656215ac90', 'request_id': 'req-619bf918-b18c-4068-88ff-5172acb5d617'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:46.395797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-619bf918-b18c-4068-88ff-5172acb5d617 admin admin] The token cache is being invalidate because project ffb685d7e14c40c8892a70656215ac90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:34:46.395797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-619bf918-b18c-4068-88ff-5172acb5d617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:46.395797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-619bf918-b18c-4068-88ff-5172acb5d617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:46.408145 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1171/2328] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:34:45 2022] DELETE /identity/v3/projects/ffb685d7e14c40c8892a70656215ac90 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:46.409795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540452d3-c605-48ac-be2c-f6fda932b136 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:46.410314 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540452d3-c605-48ac-be2c-f6fda932b136 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:46.410784 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540452d3-c605-48ac-be2c-f6fda932b136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:46.438886 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-540452d3-c605-48ac-be2c-f6fda932b136 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:46.440260 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-540452d3-c605-48ac-be2c-f6fda932b136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:46.482983 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:46.532955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1158/2329] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:34:46.541155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d1bf8-bc4a-4eb4-b6ef-09a3427577dd None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:46.541515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d1bf8-bc4a-4eb4-b6ef-09a3427577dd None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:46.541851 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d1bf8-bc4a-4eb4-b6ef-09a3427577dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:46.572049 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-ad7d1bf8-bc4a-4eb4-b6ef-09a3427577dd None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:46.573299 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ad7d1bf8-bc4a-4eb4-b6ef-09a3427577dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:46.588203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1172/2330] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:46.589745 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540452d3-c605-48ac-be2c-f6fda932b136 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:46.601289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540452d3-c605-48ac-be2c-f6fda932b136 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:46.606461 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a03568-a803-4ee8-b29e-03237c3d142e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:46.607484 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a03568-a803-4ee8-b29e-03237c3d142e admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:46.607904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a03568-a803-4ee8-b29e-03237c3d142e admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:46.608239 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a03568-a803-4ee8-b29e-03237c3d142e admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:46.609586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a03568-a803-4ee8-b29e-03237c3d142e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:34:46.609586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a03568-a803-4ee8-b29e-03237c3d142e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:46.611048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a03568-a803-4ee8-b29e-03237c3d142e admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:46.630174 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1159/2331] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:34:46.631493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d1bf8-bc4a-4eb4-b6ef-09a3427577dd None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:46.673728 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d1bf8-bc4a-4eb4-b6ef-09a3427577dd None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:46.698354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221b8eb-8f7e-4578-948c-9b7ad1e558f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:46.699054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221b8eb-8f7e-4578-948c-9b7ad1e558f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:46.700377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221b8eb-8f7e-4578-948c-9b7ad1e558f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:46.700377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221b8eb-8f7e-4578-948c-9b7ad1e558f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:46.700863 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221b8eb-8f7e-4578-948c-9b7ad1e558f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:46.702899 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221b8eb-8f7e-4578-948c-9b7ad1e558f6 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:46.844059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:46.852804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1173/2332] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:34:46 2022] POST /identity/v3/projects => generated 375 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:46.858694 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a03568-a803-4ee8-b29e-03237c3d142e admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:46.880343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a03568-a803-4ee8-b29e-03237c3d142e admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:47.014183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cda14-6a03-4812-86f6-c1de1cc5f230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:47.015180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cda14-6a03-4812-86f6-c1de1cc5f230 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:47.015573 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cda14-6a03-4812-86f6-c1de1cc5f230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:47.015945 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cda14-6a03-4812-86f6-c1de1cc5f230 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:47.017159 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4cda14-6a03-4812-86f6-c1de1cc5f230 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:47.019301 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4cda14-6a03-4812-86f6-c1de1cc5f230 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:47.122242 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1160/2333] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:34:46 2022] POST /identity/v3/users => generated 288 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:34:47.124090 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221b8eb-8f7e-4578-948c-9b7ad1e558f6 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:47.142029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221b8eb-8f7e-4578-948c-9b7ad1e558f6 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:47.150913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d2ab0-e5b2-4638-9cdf-6a818cf5f964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:47.151940 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d2ab0-e5b2-4638-9cdf-6a818cf5f964 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:47.153754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d2ab0-e5b2-4638-9cdf-6a818cf5f964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:47.154357 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d2ab0-e5b2-4638-9cdf-6a818cf5f964 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:47.157588 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d2ab0-e5b2-4638-9cdf-6a818cf5f964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:47.160238 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d2ab0-e5b2-4638-9cdf-6a818cf5f964 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:47.185900 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1174/2334] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:34:47.186129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cda14-6a03-4812-86f6-c1de1cc5f230 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:47.204043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cda14-6a03-4812-86f6-c1de1cc5f230 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:47.222115 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cbf19-06e0-4bb9-bfd3-c788f5938336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:47.223216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cbf19-06e0-4bb9-bfd3-c788f5938336 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:47.223955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cbf19-06e0-4bb9-bfd3-c788f5938336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:47.224462 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cbf19-06e0-4bb9-bfd3-c788f5938336 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:47.226138 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9cbf19-06e0-4bb9-bfd3-c788f5938336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:47.228184 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9cbf19-06e0-4bb9-bfd3-c788f5938336 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:47.285579 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1161/2335] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:34:47.286802 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d2ab0-e5b2-4638-9cdf-6a818cf5f964 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:47.305443 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d2ab0-e5b2-4638-9cdf-6a818cf5f964 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:47.320725 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e95ac-5fbd-470f-84ed-870906c9717e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:47.321656 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e95ac-5fbd-470f-84ed-870906c9717e admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:47.322052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e95ac-5fbd-470f-84ed-870906c9717e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:47.322355 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e95ac-5fbd-470f-84ed-870906c9717e admin admin] PATH_INFO: `/v3/projects/6224ef1e38214812af42576baceebee1/users/97754676e76842d2a57d1819ef4a4f85/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:47.379987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e95ac-5fbd-470f-84ed-870906c9717e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6224ef1e38214812af42576baceebee1, user_id=97754676e76842d2a57d1819ef4a4f85, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:47.384987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e95ac-5fbd-470f-84ed-870906c9717e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:47.518249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce577f-d5f6-4ccd-a16f-1882a13874b0 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:47.518648 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce577f-d5f6-4ccd-a16f-1882a13874b0 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:47.518909 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce577f-d5f6-4ccd-a16f-1882a13874b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:47.528243 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1175/2336] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:34:47 2022] PUT /identity/v3/projects/6224ef1e38214812af42576baceebee1/users/97754676e76842d2a57d1819ef4a4f85/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:47.558580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-fdce577f-d5f6-4ccd-a16f-1882a13874b0 None None] MFA Rules not processed for user `97754676e76842d2a57d1819ef4a4f85`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:47.562886 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-fdce577f-d5f6-4ccd-a16f-1882a13874b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:47.691336 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1162/2337] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:47.698973 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e95ac-5fbd-470f-84ed-870906c9717e admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:47.702953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-841e95ac-5fbd-470f-84ed-870906c9717e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:47.820695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e95ac-5fbd-470f-84ed-870906c9717e admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:47.825852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dc6e3-1750-49ed-b49c-c09fe12b025b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:47.826793 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dc6e3-1750-49ed-b49c-c09fe12b025b service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:47.827228 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dc6e3-1750-49ed-b49c-c09fe12b025b service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:47.827666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dc6e3-1750-49ed-b49c-c09fe12b025b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:47.844606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495dc6e3-1750-49ed-b49c-c09fe12b025b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:47.847236 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495dc6e3-1750-49ed-b49c-c09fe12b025b service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:47.873850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1176/2338] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:48.006033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce577f-d5f6-4ccd-a16f-1882a13874b0 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:48.010672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-fdce577f-d5f6-4ccd-a16f-1882a13874b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:48.049310 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce577f-d5f6-4ccd-a16f-1882a13874b0 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:48.053620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28cc50-6578-4b5c-83ad-487e1a60722a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:48.054319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28cc50-6578-4b5c-83ad-487e1a60722a service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:48.054613 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28cc50-6578-4b5c-83ad-487e1a60722a service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:48.054890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28cc50-6578-4b5c-83ad-487e1a60722a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:48.058194 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5b28cc50-6578-4b5c-83ad-487e1a60722a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:48.117239 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:48.235321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28cc50-6578-4b5c-83ad-487e1a60722a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:48.238356 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28cc50-6578-4b5c-83ad-487e1a60722a service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:48.269441 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1163/2339] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:56.467556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dc6e3-1750-49ed-b49c-c09fe12b025b service cinder] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:56.469444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-495dc6e3-1750-49ed-b49c-c09fe12b025b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:56.553057 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dc6e3-1750-49ed-b49c-c09fe12b025b service cinder] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:56.560315 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d444c-fd90-49bd-9511-b78731f53e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:56.561952 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d444c-fd90-49bd-9511-b78731f53e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:56.562527 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d444c-fd90-49bd-9511-b78731f53e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:56.563217 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d444c-fd90-49bd-9511-b78731f53e4f admin admin] PATH_INFO: `/v3/users/9e03db243a974515b69ea065960d3030` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:56.565338 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d444c-fd90-49bd-9511-b78731f53e4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e03db243a974515b69ea065960d3030)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:56.568428 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d444c-fd90-49bd-9511-b78731f53e4f admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:56.788330 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-965d444c-fd90-49bd-9511-b78731f53e4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e03db243a974515b69ea065960d3030'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:56.859182 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-965d444c-fd90-49bd-9511-b78731f53e4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e03db243a974515b69ea065960d3030'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:56.867536 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-965d444c-fd90-49bd-9511-b78731f53e4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e03db243a974515b69ea065960d3030'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:56.868298 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-965d444c-fd90-49bd-9511-b78731f53e4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e03db243a974515b69ea065960d3030'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:56.874792 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:56.882252 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-965d444c-fd90-49bd-9511-b78731f53e4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e03db243a974515b69ea065960d3030'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:56.888980 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1177/2340] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:34:56 2022] DELETE /identity/v3/users/9e03db243a974515b69ea065960d3030 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:56.893651 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8083fe2-f433-4044-8a9b-c7a571073349 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:56.894170 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8083fe2-f433-4044-8a9b-c7a571073349 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:56.894323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8083fe2-f433-4044-8a9b-c7a571073349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:56.912761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-c8083fe2-f433-4044-8a9b-c7a571073349 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:56.916412 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-c8083fe2-f433-4044-8a9b-c7a571073349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:56.985146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1164/2341] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:56.999287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d444c-fd90-49bd-9511-b78731f53e4f admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:57.001264 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-965d444c-fd90-49bd-9511-b78731f53e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:57.015943 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da3984-5e89-45fe-8eae-abb1e854707a None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:57.016462 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da3984-5e89-45fe-8eae-abb1e854707a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:57.016816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da3984-5e89-45fe-8eae-abb1e854707a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:57.042588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-a4da3984-5e89-45fe-8eae-abb1e854707a None None] MFA Rules not processed for user `dce92cce74c84abcbec76358c860535f`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:34:57.044552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a4da3984-5e89-45fe-8eae-abb1e854707a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:57.131712 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d444c-fd90-49bd-9511-b78731f53e4f admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:57.137515 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5cb22-1a10-42c4-99a9-bbd6312af4cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:57.138184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5cb22-1a10-42c4-99a9-bbd6312af4cf service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:57.138594 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5cb22-1a10-42c4-99a9-bbd6312af4cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:57.138857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5cb22-1a10-42c4-99a9-bbd6312af4cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:57.150896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5cb22-1a10-42c4-99a9-bbd6312af4cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:57.153411 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5cb22-1a10-42c4-99a9-bbd6312af4cf service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:57.169908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1178/2342] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:34:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:34:57.172456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1165/2343] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:34:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:34:57.768925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5cb22-1a10-42c4-99a9-bbd6312af4cf service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:57.770799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-82b5cb22-1a10-42c4-99a9-bbd6312af4cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:57.814700 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5cb22-1a10-42c4-99a9-bbd6312af4cf service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:57.826682 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e76f0a-35d0-4652-ace8-c4265b32b51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:57.828383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e76f0a-35d0-4652-ace8-c4265b32b51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:57.828991 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e76f0a-35d0-4652-ace8-c4265b32b51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:57.829652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e76f0a-35d0-4652-ace8-c4265b32b51c admin admin] PATH_INFO: `/v3/projects/29c857195a8c464f9cc94a679e5ef214` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:57.832311 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e76f0a-35d0-4652-ace8-c4265b32b51c admin admin] RBAC: Authorizing `identity:delete_project(project_id=29c857195a8c464f9cc94a679e5ef214)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:57.838249 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e76f0a-35d0-4652-ace8-c4265b32b51c admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:58.007520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-24e76f0a-35d0-4652-ace8-c4265b32b51c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29c857195a8c464f9cc94a679e5ef214', 'request_id': 'req-24e76f0a-35d0-4652-ace8-c4265b32b51c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:58.104439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-24e76f0a-35d0-4652-ace8-c4265b32b51c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29c857195a8c464f9cc94a679e5ef214', 'request_id': 'req-24e76f0a-35d0-4652-ace8-c4265b32b51c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:58.113907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-24e76f0a-35d0-4652-ace8-c4265b32b51c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29c857195a8c464f9cc94a679e5ef214', 'request_id': 'req-24e76f0a-35d0-4652-ace8-c4265b32b51c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:58.136467 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-24e76f0a-35d0-4652-ace8-c4265b32b51c admin admin] The token cache is being invalidate because project 29c857195a8c464f9cc94a679e5ef214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:34:58.137210 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-24e76f0a-35d0-4652-ace8-c4265b32b51c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:58.138179 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-24e76f0a-35d0-4652-ace8-c4265b32b51c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:58.140672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1179/2344] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:34:57 2022] DELETE /identity/v3/projects/29c857195a8c464f9cc94a679e5ef214 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:34:58.144780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da3984-5e89-45fe-8eae-abb1e854707a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:58.164505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da3984-5e89-45fe-8eae-abb1e854707a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:58.274862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6b702-7a92-499f-a1e5-d8e8b9ea1d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:58.275758 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6b702-7a92-499f-a1e5-d8e8b9ea1d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:58.276413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6b702-7a92-499f-a1e5-d8e8b9ea1d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:58.276413 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6b702-7a92-499f-a1e5-d8e8b9ea1d8b admin admin] PATH_INFO: `/v3/users/b0f2e61b180c4b74b2ce4829f5791109` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:58.277519 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6b702-7a92-499f-a1e5-d8e8b9ea1d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0f2e61b180c4b74b2ce4829f5791109)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:58.279566 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6b702-7a92-499f-a1e5-d8e8b9ea1d8b admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:58.284859 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:58.523184 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-13d6b702-7a92-499f-a1e5-d8e8b9ea1d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0f2e61b180c4b74b2ce4829f5791109'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:58.619804 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-13d6b702-7a92-499f-a1e5-d8e8b9ea1d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0f2e61b180c4b74b2ce4829f5791109'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:58.621284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-13d6b702-7a92-499f-a1e5-d8e8b9ea1d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0f2e61b180c4b74b2ce4829f5791109'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:58.659894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-13d6b702-7a92-499f-a1e5-d8e8b9ea1d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0f2e61b180c4b74b2ce4829f5791109'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:58.672840 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-13d6b702-7a92-499f-a1e5-d8e8b9ea1d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0f2e61b180c4b74b2ce4829f5791109'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:58.682443 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1166/2345] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:34:58 2022] DELETE /identity/v3/users/b0f2e61b180c4b74b2ce4829f5791109 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:59.170414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e76f0a-35d0-4652-ace8-c4265b32b51c admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:59.172660 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-24e76f0a-35d0-4652-ace8-c4265b32b51c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:59.190534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e76f0a-35d0-4652-ace8-c4265b32b51c admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:59.264249 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8e488-1b85-4a67-a8b1-de1a461a937e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:59.265117 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8e488-1b85-4a67-a8b1-de1a461a937e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:59.265425 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8e488-1b85-4a67-a8b1-de1a461a937e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:59.265702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8e488-1b85-4a67-a8b1-de1a461a937e admin admin] PATH_INFO: `/v3/projects/8a16ede4a0c143c084285a796bbcd167` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:59.266973 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8e488-1b85-4a67-a8b1-de1a461a937e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a16ede4a0c143c084285a796bbcd167)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:59.268866 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8e488-1b85-4a67-a8b1-de1a461a937e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:59.289588 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:34:59.401917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c1a8e488-1b85-4a67-a8b1-de1a461a937e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a16ede4a0c143c084285a796bbcd167', 'request_id': 'req-c1a8e488-1b85-4a67-a8b1-de1a461a937e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:59.522276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c1a8e488-1b85-4a67-a8b1-de1a461a937e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a16ede4a0c143c084285a796bbcd167', 'request_id': 'req-c1a8e488-1b85-4a67-a8b1-de1a461a937e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:59.529344 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c1a8e488-1b85-4a67-a8b1-de1a461a937e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a16ede4a0c143c084285a796bbcd167', 'request_id': 'req-c1a8e488-1b85-4a67-a8b1-de1a461a937e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:59.540006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c1a8e488-1b85-4a67-a8b1-de1a461a937e admin admin] The token cache is being invalidate because project 8a16ede4a0c143c084285a796bbcd167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:34:59.540304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c1a8e488-1b85-4a67-a8b1-de1a461a937e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:59.540963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c1a8e488-1b85-4a67-a8b1-de1a461a937e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:34:59.542606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1180/2346] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:34:59 2022] DELETE /identity/v3/projects/8a16ede4a0c143c084285a796bbcd167 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:34:59.547339 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6b702-7a92-499f-a1e5-d8e8b9ea1d8b admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:34:59.548743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-13d6b702-7a92-499f-a1e5-d8e8b9ea1d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:34:59.562159 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6b702-7a92-499f-a1e5-d8e8b9ea1d8b admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:34:59.620300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990183e-a488-44a3-b991-804c1d5142a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:34:59.621178 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990183e-a488-44a3-b991-804c1d5142a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:34:59.622011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990183e-a488-44a3-b991-804c1d5142a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:34:59.622311 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990183e-a488-44a3-b991-804c1d5142a3 admin admin] PATH_INFO: `/v3/users/21230dc4e66e45fab10ea8302d773db6` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:34:59.623540 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a990183e-a488-44a3-b991-804c1d5142a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21230dc4e66e45fab10ea8302d773db6)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:34:59.625457 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a990183e-a488-44a3-b991-804c1d5142a3 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:34:59.930800 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-a990183e-a488-44a3-b991-804c1d5142a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21230dc4e66e45fab10ea8302d773db6'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:00.007768 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-a990183e-a488-44a3-b991-804c1d5142a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21230dc4e66e45fab10ea8302d773db6'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:00.008375 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-a990183e-a488-44a3-b991-804c1d5142a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21230dc4e66e45fab10ea8302d773db6'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:00.020513 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-a990183e-a488-44a3-b991-804c1d5142a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21230dc4e66e45fab10ea8302d773db6'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:00.027321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-a990183e-a488-44a3-b991-804c1d5142a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21230dc4e66e45fab10ea8302d773db6'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:00.033871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1167/2347] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:34:59 2022] DELETE /identity/v3/users/21230dc4e66e45fab10ea8302d773db6 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:35:00.085056 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8e488-1b85-4a67-a8b1-de1a461a937e admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:00.087107 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c1a8e488-1b85-4a67-a8b1-de1a461a937e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:00.162833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8e488-1b85-4a67-a8b1-de1a461a937e admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:00.226525 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2aae9-3250-45bf-baec-745a30967800 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2192547babde4449a46f9cb37ed64524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:00.227814 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2aae9-3250-45bf-baec-745a30967800 service placement] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:00.228425 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2aae9-3250-45bf-baec-745a30967800 service placement] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:00.228425 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2aae9-3250-45bf-baec-745a30967800 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:00.230179 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0fd2aae9-3250-45bf-baec-745a30967800 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:00.242428 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2aae9-3250-45bf-baec-745a30967800 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:00.244052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2aae9-3250-45bf-baec-745a30967800 service placement] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:00.254317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1181/2348] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:35:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:35:00.558731 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990183e-a488-44a3-b991-804c1d5142a3 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:00.564346 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a990183e-a488-44a3-b991-804c1d5142a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:00.567524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:00.640502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990183e-a488-44a3-b991-804c1d5142a3 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:00.645849 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db9592-9ea0-466d-8352-bbbb99ff562a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:00.646834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db9592-9ea0-466d-8352-bbbb99ff562a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:00.647118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db9592-9ea0-466d-8352-bbbb99ff562a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:00.647397 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db9592-9ea0-466d-8352-bbbb99ff562a admin admin] PATH_INFO: `/v3/projects/6ae5869ac8df490e9489dce20e5ca9cf` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:00.648593 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db9592-9ea0-466d-8352-bbbb99ff562a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ae5869ac8df490e9489dce20e5ca9cf)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:00.650518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db9592-9ea0-466d-8352-bbbb99ff562a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:00.852276 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-64db9592-9ea0-466d-8352-bbbb99ff562a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ae5869ac8df490e9489dce20e5ca9cf', 'request_id': 'req-64db9592-9ea0-466d-8352-bbbb99ff562a'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:00.927033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-64db9592-9ea0-466d-8352-bbbb99ff562a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ae5869ac8df490e9489dce20e5ca9cf', 'request_id': 'req-64db9592-9ea0-466d-8352-bbbb99ff562a'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:00.937915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-64db9592-9ea0-466d-8352-bbbb99ff562a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ae5869ac8df490e9489dce20e5ca9cf', 'request_id': 'req-64db9592-9ea0-466d-8352-bbbb99ff562a'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:00.952485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-64db9592-9ea0-466d-8352-bbbb99ff562a admin admin] The token cache is being invalidate because project 6ae5869ac8df490e9489dce20e5ca9cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:35:00.952485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-64db9592-9ea0-466d-8352-bbbb99ff562a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:00.952978 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-64db9592-9ea0-466d-8352-bbbb99ff562a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:00.955073 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1168/2349] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:35:00 2022] DELETE /identity/v3/projects/6ae5869ac8df490e9489dce20e5ca9cf => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:35:03.130777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01961d9-4f78-48c3-a0aa-3e80df921807 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:03.131361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01961d9-4f78-48c3-a0aa-3e80df921807 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:03.131960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01961d9-4f78-48c3-a0aa-3e80df921807 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:03.133559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1182/2350] 10.222.0.78 () {54 vars in 921 bytes} [Wed Jul 20 15:35:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:35:03.142869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99f352-1c3e-4681-a69c-79fd086ef6fe None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:03.143402 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99f352-1c3e-4681-a69c-79fd086ef6fe None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:03.143641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99f352-1c3e-4681-a69c-79fd086ef6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:03.162213 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-9b99f352-1c3e-4681-a69c-79fd086ef6fe None None] MFA Rules not processed for user `a4a24d5a375945779fa626fae25ac237`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:03.163691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-9b99f352-1c3e-4681-a69c-79fd086ef6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:03.250343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1169/2351] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Jul 20 15:35:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:03.265542 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01961d9-4f78-48c3-a0aa-3e80df921807 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:03.267572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f01961d9-4f78-48c3-a0aa-3e80df921807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:03.316755 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:03.339085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01961d9-4f78-48c3-a0aa-3e80df921807 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:03.343727 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4455e2-31eb-404e-bf20-f09fa8fbc80a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6c13596c2df438294bdac7c52ae8cb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:03.344706 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4455e2-31eb-404e-bf20-f09fa8fbc80a service glance] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:03.345009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4455e2-31eb-404e-bf20-f09fa8fbc80a service glance] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:03.345420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4455e2-31eb-404e-bf20-f09fa8fbc80a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:03.362047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4455e2-31eb-404e-bf20-f09fa8fbc80a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:03.363565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4455e2-31eb-404e-bf20-f09fa8fbc80a service glance] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:03.375848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1183/2352] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:35:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:35:03.661083 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fe537-5705-4677-8a19-2b1d306856f7 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:03.661373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fe537-5705-4677-8a19-2b1d306856f7 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:03.661651 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fe537-5705-4677-8a19-2b1d306856f7 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:03.663363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1170/2353] 10.222.0.78 () {54 vars in 921 bytes} [Wed Jul 20 15:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:35:03.668362 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84380ad2-5445-475a-9742-e065a2cd55cf None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:03.668651 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84380ad2-5445-475a-9742-e065a2cd55cf None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:03.668973 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84380ad2-5445-475a-9742-e065a2cd55cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:03.688412 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-84380ad2-5445-475a-9742-e065a2cd55cf None None] MFA Rules not processed for user `b57530f18d9c4d328f8014db736d8de9`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:03.689642 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-84380ad2-5445-475a-9742-e065a2cd55cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:03.749842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1184/2354] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Jul 20 15:35:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:03.772826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fe537-5705-4677-8a19-2b1d306856f7 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:03.772826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-873fe537-5705-4677-8a19-2b1d306856f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:03.779935 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fe537-5705-4677-8a19-2b1d306856f7 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:03.783577 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c52e8-226a-4124-bceb-680d0ef87e00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:03.784226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c52e8-226a-4124-bceb-680d0ef87e00 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:03.784511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c52e8-226a-4124-bceb-680d0ef87e00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:03.784789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c52e8-226a-4124-bceb-680d0ef87e00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:03.794969 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c52e8-226a-4124-bceb-680d0ef87e00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:03.796353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c52e8-226a-4124-bceb-680d0ef87e00 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:03.810541 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1171/2355] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:35:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:35:03.849066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84380ad2-5445-475a-9742-e065a2cd55cf None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:03.850855 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-84380ad2-5445-475a-9742-e065a2cd55cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:03.953546 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84380ad2-5445-475a-9742-e065a2cd55cf None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:03.959107 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35aac02-5ba6-47cd-9007-6a67ba21b023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:03.960276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aac02-5ba6-47cd-9007-6a67ba21b023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:03.960576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aac02-5ba6-47cd-9007-6a67ba21b023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:03.960857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aac02-5ba6-47cd-9007-6a67ba21b023 admin admin] PATH_INFO: `/v3/users/4f837f8db5504da08424e1ec2ed94f6f` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:03.962563 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35aac02-5ba6-47cd-9007-6a67ba21b023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f837f8db5504da08424e1ec2ed94f6f)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:03.964808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35aac02-5ba6-47cd-9007-6a67ba21b023 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:03.966836 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8386227-dfbe-44a8-8d22-c7441a73df88 None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:03.967314 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8386227-dfbe-44a8-8d22-c7441a73df88 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:03.967984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8386227-dfbe-44a8-8d22-c7441a73df88 None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:03.969592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1172/2356] 10.222.0.78 () {54 vars in 921 bytes} [Wed Jul 20 15:35:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:35:03.981003 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c60a13-38fa-41ea-8e46-671fe6f5e349 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:03.981003 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c60a13-38fa-41ea-8e46-671fe6f5e349 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:03.981003 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c60a13-38fa-41ea-8e46-671fe6f5e349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:04.003744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-85c60a13-38fa-41ea-8e46-671fe6f5e349 None None] MFA Rules not processed for user `a4a24d5a375945779fa626fae25ac237`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:04.004782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-85c60a13-38fa-41ea-8e46-671fe6f5e349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:04.016245 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1173/2357] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Jul 20 15:35:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:35:04.027624 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c60a13-38fa-41ea-8e46-671fe6f5e349 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:04.030040 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-85c60a13-38fa-41ea-8e46-671fe6f5e349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:04.039645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:04.043983 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c60a13-38fa-41ea-8e46-671fe6f5e349 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:04.047699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535c07d-33d2-4811-96b0-d9f327557132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:04.048344 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535c07d-33d2-4811-96b0-d9f327557132 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:04.048623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535c07d-33d2-4811-96b0-d9f327557132 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:04.048902 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535c07d-33d2-4811-96b0-d9f327557132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:04.060530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535c07d-33d2-4811-96b0-d9f327557132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:04.061972 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535c07d-33d2-4811-96b0-d9f327557132 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:04.072699 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1174/2358] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:35:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:35:04.233377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b35aac02-5ba6-47cd-9007-6a67ba21b023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f837f8db5504da08424e1ec2ed94f6f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:04.326565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b35aac02-5ba6-47cd-9007-6a67ba21b023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f837f8db5504da08424e1ec2ed94f6f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:04.334978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b35aac02-5ba6-47cd-9007-6a67ba21b023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f837f8db5504da08424e1ec2ed94f6f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:04.335691 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b35aac02-5ba6-47cd-9007-6a67ba21b023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f837f8db5504da08424e1ec2ed94f6f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:04.348123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b35aac02-5ba6-47cd-9007-6a67ba21b023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f837f8db5504da08424e1ec2ed94f6f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:04.354183 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1185/2359] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:35:03 2022] DELETE /identity/v3/users/4f837f8db5504da08424e1ec2ed94f6f => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:35:04.359333 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c1d83-e2dc-4738-b38d-aa8aa314ebee None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:04.359840 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c1d83-e2dc-4738-b38d-aa8aa314ebee None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:04.360001 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c1d83-e2dc-4738-b38d-aa8aa314ebee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:04.378706 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-e45c1d83-e2dc-4738-b38d-aa8aa314ebee None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:04.379642 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e45c1d83-e2dc-4738-b38d-aa8aa314ebee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:04.447732 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1175/2360] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:04.458136 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35aac02-5ba6-47cd-9007-6a67ba21b023 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:04.459629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b35aac02-5ba6-47cd-9007-6a67ba21b023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:04.470847 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:04.522079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35aac02-5ba6-47cd-9007-6a67ba21b023 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:04.526196 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352635b4-2f4b-4b2a-8021-bc7641f06822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:04.526892 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352635b4-2f4b-4b2a-8021-bc7641f06822 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:04.527176 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352635b4-2f4b-4b2a-8021-bc7641f06822 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:04.528007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352635b4-2f4b-4b2a-8021-bc7641f06822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:04.537745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352635b4-2f4b-4b2a-8021-bc7641f06822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:04.539222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352635b4-2f4b-4b2a-8021-bc7641f06822 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:04.549961 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1186/2361] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:35:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:35:05.072215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c1d83-e2dc-4738-b38d-aa8aa314ebee None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:05.073747 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e45c1d83-e2dc-4738-b38d-aa8aa314ebee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:05.095449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c1d83-e2dc-4738-b38d-aa8aa314ebee None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:05.103823 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb89a09-90e3-414a-aa50-1efa53c7d5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:05.105672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb89a09-90e3-414a-aa50-1efa53c7d5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:05.118062 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb89a09-90e3-414a-aa50-1efa53c7d5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:05.119015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb89a09-90e3-414a-aa50-1efa53c7d5b4 admin admin] PATH_INFO: `/v3/projects/aa42974b2b0342fca28da5a3e98bba41` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:05.122738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb89a09-90e3-414a-aa50-1efa53c7d5b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa42974b2b0342fca28da5a3e98bba41)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:05.124822 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb89a09-90e3-414a-aa50-1efa53c7d5b4 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:05.253443 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ccb89a09-90e3-414a-aa50-1efa53c7d5b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa42974b2b0342fca28da5a3e98bba41', 'request_id': 'req-ccb89a09-90e3-414a-aa50-1efa53c7d5b4'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:05.338528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ccb89a09-90e3-414a-aa50-1efa53c7d5b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa42974b2b0342fca28da5a3e98bba41', 'request_id': 'req-ccb89a09-90e3-414a-aa50-1efa53c7d5b4'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:05.347758 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ccb89a09-90e3-414a-aa50-1efa53c7d5b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa42974b2b0342fca28da5a3e98bba41', 'request_id': 'req-ccb89a09-90e3-414a-aa50-1efa53c7d5b4'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:05.366764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ccb89a09-90e3-414a-aa50-1efa53c7d5b4 admin admin] The token cache is being invalidate because project aa42974b2b0342fca28da5a3e98bba41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:35:05.368112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ccb89a09-90e3-414a-aa50-1efa53c7d5b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:05.369204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ccb89a09-90e3-414a-aa50-1efa53c7d5b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:05.372323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1176/2362] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:35:05 2022] DELETE /identity/v3/projects/aa42974b2b0342fca28da5a3e98bba41 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:35:05.377274 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec1130-621b-435b-ba4a-1c122526859a None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:05.377752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec1130-621b-435b-ba4a-1c122526859a None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:05.377850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec1130-621b-435b-ba4a-1c122526859a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:05.424500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-74ec1130-621b-435b-ba4a-1c122526859a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:05.425787 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-74ec1130-621b-435b-ba4a-1c122526859a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:05.569815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1187/2363] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:05.581924 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc36150-b5ac-4722-b173-747b2c0998d0 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:05.582228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc36150-b5ac-4722-b173-747b2c0998d0 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:05.582580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc36150-b5ac-4722-b173-747b2c0998d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:05.606707 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-2fc36150-b5ac-4722-b173-747b2c0998d0 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:05.608045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-2fc36150-b5ac-4722-b173-747b2c0998d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:05.620108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1177/2364] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:35:05.627458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec1130-621b-435b-ba4a-1c122526859a None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:05.637406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec1130-621b-435b-ba4a-1c122526859a None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:05.642839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a55b6-ec6a-437d-a98a-41addabb1643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:05.643814 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a55b6-ec6a-437d-a98a-41addabb1643 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:05.644259 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a55b6-ec6a-437d-a98a-41addabb1643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:05.644741 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a55b6-ec6a-437d-a98a-41addabb1643 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:05.645752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a55b6-ec6a-437d-a98a-41addabb1643 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:35:05.646215 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a55b6-ec6a-437d-a98a-41addabb1643 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:05.648232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a55b6-ec6a-437d-a98a-41addabb1643 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:05.664820 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1188/2365] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:35:05.666183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc36150-b5ac-4722-b173-747b2c0998d0 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:05.677584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc36150-b5ac-4722-b173-747b2c0998d0 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:05.688555 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188650d7-3d1b-4210-9783-096e994a9a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:05.689331 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188650d7-3d1b-4210-9783-096e994a9a35 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:05.689719 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188650d7-3d1b-4210-9783-096e994a9a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:05.690085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188650d7-3d1b-4210-9783-096e994a9a35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:05.693434 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188650d7-3d1b-4210-9783-096e994a9a35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:05.695564 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188650d7-3d1b-4210-9783-096e994a9a35 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:05.698222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff7f6a-8554-448e-8c79-f4429d4ff85f None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:05.698736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff7f6a-8554-448e-8c79-f4429d4ff85f None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:05.699218 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff7f6a-8554-448e-8c79-f4429d4ff85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:05.722116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-47ff7f6a-8554-448e-8c79-f4429d4ff85f None None] MFA Rules not processed for user `4e75a82cc7e94702ba0080fc65d6eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:05.723451 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-47ff7f6a-8554-448e-8c79-f4429d4ff85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:05.790692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1189/2366] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Jul 20 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:05.800919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1178/2367] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:35:05 2022] POST /identity/v3/projects => generated 395 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:35:05.804304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff7f6a-8554-448e-8c79-f4429d4ff85f None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:05.806826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188650d7-3d1b-4210-9783-096e994a9a35 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:05.809219 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-188650d7-3d1b-4210-9783-096e994a9a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:05.819112 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff7f6a-8554-448e-8c79-f4429d4ff85f None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:05.871794 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:05.873211 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:05.891019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fb95e-92e3-489b-911e-0db23e367d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:05.891702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fb95e-92e3-489b-911e-0db23e367d40 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:05.891988 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fb95e-92e3-489b-911e-0db23e367d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:05.892270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fb95e-92e3-489b-911e-0db23e367d40 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:05.892978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32fb95e-92e3-489b-911e-0db23e367d40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:05.894964 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32fb95e-92e3-489b-911e-0db23e367d40 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:05.899225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188650d7-3d1b-4210-9783-096e994a9a35 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:05.905910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15cb75-54a4-4b47-9e82-63b4d1744ea5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0d719c0eb04a3b8a228c6ce70fdf98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:05.906763 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15cb75-54a4-4b47-9e82-63b4d1744ea5 service swift] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:05.907046 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15cb75-54a4-4b47-9e82-63b4d1744ea5 service swift] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:05.907321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15cb75-54a4-4b47-9e82-63b4d1744ea5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:05.966409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15cb75-54a4-4b47-9e82-63b4d1744ea5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:05.968058 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15cb75-54a4-4b47-9e82-63b4d1744ea5 service swift] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:05.976534 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1179/2368] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Jul 20 15:35:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 15:35:06.025985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1190/2369] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:35:05 2022] POST /identity/v3/users => generated 298 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:35:06.030672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15cb75-54a4-4b47-9e82-63b4d1744ea5 service swift] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:06.041477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15cb75-54a4-4b47-9e82-63b4d1744ea5 service swift] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:06.046945 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bddd3-03c1-4f5e-804c-f637b39e7886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:06.047637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bddd3-03c1-4f5e-804c-f637b39e7886 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:06.048081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bddd3-03c1-4f5e-804c-f637b39e7886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:06.048227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bddd3-03c1-4f5e-804c-f637b39e7886 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:06.048964 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436bddd3-03c1-4f5e-804c-f637b39e7886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:06.050632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436bddd3-03c1-4f5e-804c-f637b39e7886 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:06.079958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1180/2370] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:35:06.081159 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fb95e-92e3-489b-911e-0db23e367d40 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:06.097828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fb95e-92e3-489b-911e-0db23e367d40 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:06.107170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c2e81-a23d-41b3-81ac-5fed15e42fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:06.108184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c2e81-a23d-41b3-81ac-5fed15e42fed admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:06.108533 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c2e81-a23d-41b3-81ac-5fed15e42fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:06.108880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c2e81-a23d-41b3-81ac-5fed15e42fed admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:06.109966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c2e81-a23d-41b3-81ac-5fed15e42fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:06.125242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c2e81-a23d-41b3-81ac-5fed15e42fed admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:06.158571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bddd3-03c1-4f5e-804c-f637b39e7886 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:06.158863 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1191/2371] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:35:06.171254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bddd3-03c1-4f5e-804c-f637b39e7886 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:06.190530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10f638-2769-4082-846f-55ec6a658827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:06.191441 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10f638-2769-4082-846f-55ec6a658827 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:06.191722 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10f638-2769-4082-846f-55ec6a658827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:06.191998 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10f638-2769-4082-846f-55ec6a658827 admin admin] PATH_INFO: `/v3/projects/3319901db85942b4a0bc324ba32e5f5e/users/a6b07faccd44480580cbd8fd4989621e/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:06.222601 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10f638-2769-4082-846f-55ec6a658827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3319901db85942b4a0bc324ba32e5f5e, user_id=a6b07faccd44480580cbd8fd4989621e, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:06.229001 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10f638-2769-4082-846f-55ec6a658827 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:06.313096 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1181/2372] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:35:06 2022] PUT /identity/v3/projects/3319901db85942b4a0bc324ba32e5f5e/users/a6b07faccd44480580cbd8fd4989621e/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:35:06.323538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af5784-e5b8-4e76-b667-24eac3c29e14 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:06.324648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af5784-e5b8-4e76-b667-24eac3c29e14 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:06.325025 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af5784-e5b8-4e76-b667-24eac3c29e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:06.355922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-17af5784-e5b8-4e76-b667-24eac3c29e14 None None] MFA Rules not processed for user `a6b07faccd44480580cbd8fd4989621e`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:06.357548 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-17af5784-e5b8-4e76-b667-24eac3c29e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:06.435043 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1192/2373] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:06.606217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10f638-2769-4082-846f-55ec6a658827 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:06.608149 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-3d10f638-2769-4082-846f-55ec6a658827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:06.666886 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10f638-2769-4082-846f-55ec6a658827 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:06.670590 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85509220-1e7c-4c11-9dda-dc8a5172e3bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:06.671231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85509220-1e7c-4c11-9dda-dc8a5172e3bf service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:06.671512 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85509220-1e7c-4c11-9dda-dc8a5172e3bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:06.671786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85509220-1e7c-4c11-9dda-dc8a5172e3bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:06.680757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85509220-1e7c-4c11-9dda-dc8a5172e3bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:06.682287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85509220-1e7c-4c11-9dda-dc8a5172e3bf service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:06.691976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1182/2374] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:35:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:35:14.634771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af5784-e5b8-4e76-b667-24eac3c29e14 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:14.636192 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-17af5784-e5b8-4e76-b667-24eac3c29e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:14.701112 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af5784-e5b8-4e76-b667-24eac3c29e14 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:14.705598 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b0b15-1335-49a6-b784-4f47cc2ca741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:14.706498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b0b15-1335-49a6-b784-4f47cc2ca741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:14.706782 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b0b15-1335-49a6-b784-4f47cc2ca741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:14.707061 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b0b15-1335-49a6-b784-4f47cc2ca741 admin admin] PATH_INFO: `/v3/users/d3e4a363addc490ab87a11aa7e158460` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:14.708253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b0b15-1335-49a6-b784-4f47cc2ca741 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3e4a363addc490ab87a11aa7e158460)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:14.710168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b0b15-1335-49a6-b784-4f47cc2ca741 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:14.908583 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-bb3b0b15-1335-49a6-b784-4f47cc2ca741 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3e4a363addc490ab87a11aa7e158460'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:14.912567 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:14.989295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-bb3b0b15-1335-49a6-b784-4f47cc2ca741 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3e4a363addc490ab87a11aa7e158460'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:14.997233 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-bb3b0b15-1335-49a6-b784-4f47cc2ca741 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3e4a363addc490ab87a11aa7e158460'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:14.998705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-bb3b0b15-1335-49a6-b784-4f47cc2ca741 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3e4a363addc490ab87a11aa7e158460'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:15.010872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-bb3b0b15-1335-49a6-b784-4f47cc2ca741 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3e4a363addc490ab87a11aa7e158460'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:15.017610 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1193/2375] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:35:14 2022] DELETE /identity/v3/users/d3e4a363addc490ab87a11aa7e158460 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:35:15.022475 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90fab9-2682-47ed-bb28-053c3751553f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:15.029503 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90fab9-2682-47ed-bb28-053c3751553f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:15.029503 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90fab9-2682-47ed-bb28-053c3751553f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:15.060781 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-9f90fab9-2682-47ed-bb28-053c3751553f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:15.062370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-9f90fab9-2682-47ed-bb28-053c3751553f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:15.151419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:15.177269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1183/2376] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:15.184502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b0b15-1335-49a6-b784-4f47cc2ca741 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:15.191116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-bb3b0b15-1335-49a6-b784-4f47cc2ca741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:15.307693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b0b15-1335-49a6-b784-4f47cc2ca741 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:15.312411 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6063f-6c5c-40a9-93df-55dfd3c60567 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:15.313161 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6063f-6c5c-40a9-93df-55dfd3c60567 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:15.313448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6063f-6c5c-40a9-93df-55dfd3c60567 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:15.313751 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6063f-6c5c-40a9-93df-55dfd3c60567 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:15.324877 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb6063f-6c5c-40a9-93df-55dfd3c60567 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:15.326525 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb6063f-6c5c-40a9-93df-55dfd3c60567 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:15.336674 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1194/2377] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:35:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:35:15.917936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90fab9-2682-47ed-bb28-053c3751553f None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:15.919823 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-9f90fab9-2682-47ed-bb28-053c3751553f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:15.935031 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90fab9-2682-47ed-bb28-053c3751553f None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:15.940662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399c987-b2b0-4e82-9221-a3b9ea4a64ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:15.941717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399c987-b2b0-4e82-9221-a3b9ea4a64ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:15.942125 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399c987-b2b0-4e82-9221-a3b9ea4a64ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:15.942529 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399c987-b2b0-4e82-9221-a3b9ea4a64ec admin admin] PATH_INFO: `/v3/projects/f20de04a5f9f4355bbad81e50380a12f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:15.943887 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399c987-b2b0-4e82-9221-a3b9ea4a64ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=f20de04a5f9f4355bbad81e50380a12f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:15.946087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399c987-b2b0-4e82-9221-a3b9ea4a64ec admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:16.270462 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-a399c987-b2b0-4e82-9221-a3b9ea4a64ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f20de04a5f9f4355bbad81e50380a12f', 'request_id': 'req-a399c987-b2b0-4e82-9221-a3b9ea4a64ec'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:16.425332 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-a399c987-b2b0-4e82-9221-a3b9ea4a64ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f20de04a5f9f4355bbad81e50380a12f', 'request_id': 'req-a399c987-b2b0-4e82-9221-a3b9ea4a64ec'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:16.434078 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-a399c987-b2b0-4e82-9221-a3b9ea4a64ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f20de04a5f9f4355bbad81e50380a12f', 'request_id': 'req-a399c987-b2b0-4e82-9221-a3b9ea4a64ec'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:16.447562 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-a399c987-b2b0-4e82-9221-a3b9ea4a64ec admin admin] The token cache is being invalidate because project f20de04a5f9f4355bbad81e50380a12f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:35:16.448310 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-a399c987-b2b0-4e82-9221-a3b9ea4a64ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:16.449821 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-a399c987-b2b0-4e82-9221-a3b9ea4a64ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:16.462317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1184/2378] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:35:15 2022] DELETE /identity/v3/projects/f20de04a5f9f4355bbad81e50380a12f => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:35:16.908042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6063f-6c5c-40a9-93df-55dfd3c60567 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:16.911411 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6bb6063f-6c5c-40a9-93df-55dfd3c60567 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:16.924723 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6063f-6c5c-40a9-93df-55dfd3c60567 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:16.998796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05d155-ae39-4f10-89b4-78fb89197701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:17.000167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05d155-ae39-4f10-89b4-78fb89197701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:17.000748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05d155-ae39-4f10-89b4-78fb89197701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:17.001208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05d155-ae39-4f10-89b4-78fb89197701 admin admin] PATH_INFO: `/v3/users/a6b07faccd44480580cbd8fd4989621e` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:17.002565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05d155-ae39-4f10-89b4-78fb89197701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6b07faccd44480580cbd8fd4989621e)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:17.004593 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05d155-ae39-4f10-89b4-78fb89197701 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:17.227148 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-cc05d155-ae39-4f10-89b4-78fb89197701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6b07faccd44480580cbd8fd4989621e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:17.298637 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-cc05d155-ae39-4f10-89b4-78fb89197701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6b07faccd44480580cbd8fd4989621e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:17.310066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-cc05d155-ae39-4f10-89b4-78fb89197701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6b07faccd44480580cbd8fd4989621e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:17.312997 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-cc05d155-ae39-4f10-89b4-78fb89197701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6b07faccd44480580cbd8fd4989621e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:17.333029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:17.342843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-cc05d155-ae39-4f10-89b4-78fb89197701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6b07faccd44480580cbd8fd4989621e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:17.375587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c333672-6095-41ce-be2b-fd2ee0cb794c None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:17.376191 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c333672-6095-41ce-be2b-fd2ee0cb794c None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:17.376191 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c333672-6095-41ce-be2b-fd2ee0cb794c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:17.377880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1195/2379] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:35:16 2022] DELETE /identity/v3/users/a6b07faccd44480580cbd8fd4989621e => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:35:17.408854 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-1c333672-6095-41ce-be2b-fd2ee0cb794c None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:17.419896 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-1c333672-6095-41ce-be2b-fd2ee0cb794c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:17.504505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1185/2380] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:17.517086 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05d155-ae39-4f10-89b4-78fb89197701 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:17.518877 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-cc05d155-ae39-4f10-89b4-78fb89197701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:17.615361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05d155-ae39-4f10-89b4-78fb89197701 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:17.619806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d568007-f103-4f14-b2e9-591cb807fa2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:17.625799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d568007-f103-4f14-b2e9-591cb807fa2b service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:17.625799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d568007-f103-4f14-b2e9-591cb807fa2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:17.625799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d568007-f103-4f14-b2e9-591cb807fa2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:17.632469 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d568007-f103-4f14-b2e9-591cb807fa2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:17.633922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d568007-f103-4f14-b2e9-591cb807fa2b service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:17.643935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1196/2381] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:35:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:35:18.204384 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c333672-6095-41ce-be2b-fd2ee0cb794c None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:18.205801 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-1c333672-6095-41ce-be2b-fd2ee0cb794c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:18.218178 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c333672-6095-41ce-be2b-fd2ee0cb794c None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:18.222831 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05f4b8-81aa-48fa-9814-aa9edff4d5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:18.223745 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05f4b8-81aa-48fa-9814-aa9edff4d5bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:18.224049 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05f4b8-81aa-48fa-9814-aa9edff4d5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:18.224284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05f4b8-81aa-48fa-9814-aa9edff4d5bf admin admin] PATH_INFO: `/v3/projects/3319901db85942b4a0bc324ba32e5f5e` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:18.225501 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05f4b8-81aa-48fa-9814-aa9edff4d5bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3319901db85942b4a0bc324ba32e5f5e)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:18.227547 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05f4b8-81aa-48fa-9814-aa9edff4d5bf admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:18.359768 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-dc05f4b8-81aa-48fa-9814-aa9edff4d5bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3319901db85942b4a0bc324ba32e5f5e', 'request_id': 'req-dc05f4b8-81aa-48fa-9814-aa9edff4d5bf'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:18.475077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-dc05f4b8-81aa-48fa-9814-aa9edff4d5bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3319901db85942b4a0bc324ba32e5f5e', 'request_id': 'req-dc05f4b8-81aa-48fa-9814-aa9edff4d5bf'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:18.482845 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-dc05f4b8-81aa-48fa-9814-aa9edff4d5bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3319901db85942b4a0bc324ba32e5f5e', 'request_id': 'req-dc05f4b8-81aa-48fa-9814-aa9edff4d5bf'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:18.494082 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-dc05f4b8-81aa-48fa-9814-aa9edff4d5bf admin admin] The token cache is being invalidate because project 3319901db85942b4a0bc324ba32e5f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:35:18.494401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-dc05f4b8-81aa-48fa-9814-aa9edff4d5bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:18.495057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-dc05f4b8-81aa-48fa-9814-aa9edff4d5bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:18.496679 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1186/2382] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:35:18 2022] DELETE /identity/v3/projects/3319901db85942b4a0bc324ba32e5f5e => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:35:18.502407 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1eb84-df5c-4d3e-ac2f-ea09c9489b6e None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:18.505466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1eb84-df5c-4d3e-ac2f-ea09c9489b6e None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:18.505466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1eb84-df5c-4d3e-ac2f-ea09c9489b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:18.521332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-24f1eb84-df5c-4d3e-ac2f-ea09c9489b6e None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:18.522669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-24f1eb84-df5c-4d3e-ac2f-ea09c9489b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:18.601006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1197/2383] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:18.611733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6ebff-ce01-4334-83f3-1aa81392554e None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:18.612091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6ebff-ce01-4334-83f3-1aa81392554e None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:18.612377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6ebff-ce01-4334-83f3-1aa81392554e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:18.619023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:18.633582 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-18a6ebff-ce01-4334-83f3-1aa81392554e None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:18.635386 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-18a6ebff-ce01-4334-83f3-1aa81392554e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:18.644373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1187/2384] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:18.649538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1eb84-df5c-4d3e-ac2f-ea09c9489b6e None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:18.672899 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1eb84-df5c-4d3e-ac2f-ea09c9489b6e None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:18.678675 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d569f36-ee0d-4c21-9a07-a52ed742c478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:18.679353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d569f36-ee0d-4c21-9a07-a52ed742c478 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:18.679640 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d569f36-ee0d-4c21-9a07-a52ed742c478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:18.679918 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d569f36-ee0d-4c21-9a07-a52ed742c478 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:18.680653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d569f36-ee0d-4c21-9a07-a52ed742c478 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:35:18.680961 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d569f36-ee0d-4c21-9a07-a52ed742c478 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:18.682594 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d569f36-ee0d-4c21-9a07-a52ed742c478 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:18.699264 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1198/2385] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:35:18.700860 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6ebff-ce01-4334-83f3-1aa81392554e None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:18.713035 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6ebff-ce01-4334-83f3-1aa81392554e None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:18.719629 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7dbe4e-5b86-47aa-87fb-797a9fdc1ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:18.720484 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dbe4e-5b86-47aa-87fb-797a9fdc1ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:18.720779 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dbe4e-5b86-47aa-87fb-797a9fdc1ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:18.721204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dbe4e-5b86-47aa-87fb-797a9fdc1ed3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:18.722092 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7dbe4e-5b86-47aa-87fb-797a9fdc1ed3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:18.724085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7dbe4e-5b86-47aa-87fb-797a9fdc1ed3 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:18.871129 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d569f36-ee0d-4c21-9a07-a52ed742c478 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:18.873795 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0d569f36-ee0d-4c21-9a07-a52ed742c478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:18.880160 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1188/2386] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:35:18 2022] POST /identity/v3/projects => generated 371 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:35:18.884096 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7dbe4e-5b86-47aa-87fb-797a9fdc1ed3 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:18.891070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d569f36-ee0d-4c21-9a07-a52ed742c478 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:18.979965 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cf2c0-b490-4170-8329-a4ac9e0bbdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:18.984024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cf2c0-b490-4170-8329-a4ac9e0bbdf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:18.984024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cf2c0-b490-4170-8329-a4ac9e0bbdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:18.984024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cf2c0-b490-4170-8329-a4ac9e0bbdf4 admin admin] PATH_INFO: `/v3/users/97754676e76842d2a57d1819ef4a4f85` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:18.984024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042cf2c0-b490-4170-8329-a4ac9e0bbdf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97754676e76842d2a57d1819ef4a4f85)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:18.985854 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042cf2c0-b490-4170-8329-a4ac9e0bbdf4 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:18.992075 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7dbe4e-5b86-47aa-87fb-797a9fdc1ed3 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:18.996985 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143dd95-abb6-4c50-af09-f1c12de181e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:18.997659 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143dd95-abb6-4c50-af09-f1c12de181e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:18.997946 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143dd95-abb6-4c50-af09-f1c12de181e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:18.998298 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143dd95-abb6-4c50-af09-f1c12de181e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:18.999085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143dd95-abb6-4c50-af09-f1c12de181e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:19.001059 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143dd95-abb6-4c50-af09-f1c12de181e7 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:19.021582 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:19.125006 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1189/2387] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:35:18 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:35:19.126063 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143dd95-abb6-4c50-af09-f1c12de181e7 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:19.140592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143dd95-abb6-4c50-af09-f1c12de181e7 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:19.155881 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd3aa2-7246-455c-99ae-119e2f513233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:19.156658 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd3aa2-7246-455c-99ae-119e2f513233 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:19.156959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd3aa2-7246-455c-99ae-119e2f513233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:19.157248 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd3aa2-7246-455c-99ae-119e2f513233 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:19.158147 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd3aa2-7246-455c-99ae-119e2f513233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:19.160019 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd3aa2-7246-455c-99ae-119e2f513233 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:19.186955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1190/2388] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:35:19.193329 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd3aa2-7246-455c-99ae-119e2f513233 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:19.220601 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd3aa2-7246-455c-99ae-119e2f513233 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:19.236044 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27728767-91a0-4188-89b0-d07a15290c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:19.243622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27728767-91a0-4188-89b0-d07a15290c18 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:19.243622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27728767-91a0-4188-89b0-d07a15290c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:19.243622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27728767-91a0-4188-89b0-d07a15290c18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:19.243622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27728767-91a0-4188-89b0-d07a15290c18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:19.243622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27728767-91a0-4188-89b0-d07a15290c18 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:19.270069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1191/2389] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:35:19.279449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27728767-91a0-4188-89b0-d07a15290c18 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:19.301420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-042cf2c0-b490-4170-8329-a4ac9e0bbdf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97754676e76842d2a57d1819ef4a4f85'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:19.304670 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27728767-91a0-4188-89b0-d07a15290c18 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:19.313071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed78ecd-c3e9-44eb-94a2-d1b932b05778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:19.313071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed78ecd-c3e9-44eb-94a2-d1b932b05778 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:19.318045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed78ecd-c3e9-44eb-94a2-d1b932b05778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:19.318045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed78ecd-c3e9-44eb-94a2-d1b932b05778 admin admin] PATH_INFO: `/v3/projects/09d92d320f8748c58aca3fd6a8776540/users/ec2e023235a343dd81754211ee102f3c/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:19.346120 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed78ecd-c3e9-44eb-94a2-d1b932b05778 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d92d320f8748c58aca3fd6a8776540, user_id=ec2e023235a343dd81754211ee102f3c, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:19.350652 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed78ecd-c3e9-44eb-94a2-d1b932b05778 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:19.493099 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-042cf2c0-b490-4170-8329-a4ac9e0bbdf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97754676e76842d2a57d1819ef4a4f85'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:19.499644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1192/2390] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:35:19 2022] PUT /identity/v3/projects/09d92d320f8748c58aca3fd6a8776540/users/ec2e023235a343dd81754211ee102f3c/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:35:19.505253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-042cf2c0-b490-4170-8329-a4ac9e0bbdf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97754676e76842d2a57d1819ef4a4f85'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:19.505253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-042cf2c0-b490-4170-8329-a4ac9e0bbdf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97754676e76842d2a57d1819ef4a4f85'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:19.508960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90565cc0-4849-40d9-95cb-672bbbfc9d06 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:19.509317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90565cc0-4849-40d9-95cb-672bbbfc9d06 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:19.509602 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90565cc0-4849-40d9-95cb-672bbbfc9d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:19.515839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-042cf2c0-b490-4170-8329-a4ac9e0bbdf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97754676e76842d2a57d1819ef4a4f85'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:19.522165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1199/2391] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:35:18 2022] DELETE /identity/v3/users/97754676e76842d2a57d1819ef4a4f85 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:35:19.526504 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed26149-c210-4261-91b5-2fee5c409c7c None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:19.526742 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-90565cc0-4849-40d9-95cb-672bbbfc9d06 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:19.526858 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed26149-c210-4261-91b5-2fee5c409c7c None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:19.527074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed26149-c210-4261-91b5-2fee5c409c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:19.528480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-90565cc0-4849-40d9-95cb-672bbbfc9d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:19.555863 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-7ed26149-c210-4261-91b5-2fee5c409c7c None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:19.557265 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-7ed26149-c210-4261-91b5-2fee5c409c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:19.604785 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1193/2392] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:35:19.622684 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90565cc0-4849-40d9-95cb-672bbbfc9d06 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:19.622684 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-90565cc0-4849-40d9-95cb-672bbbfc9d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:19.637071 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1200/2393] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:35:19.647937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed26149-c210-4261-91b5-2fee5c409c7c None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:19.650935 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:19.651373 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-7ed26149-c210-4261-91b5-2fee5c409c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:19.733342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed26149-c210-4261-91b5-2fee5c409c7c None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:19.734072 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90565cc0-4849-40d9-95cb-672bbbfc9d06 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:19.737557 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876236ce-fbd7-4940-b7a8-c43f4f14982f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:19.738682 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876236ce-fbd7-4940-b7a8-c43f4f14982f service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:19.739373 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876236ce-fbd7-4940-b7a8-c43f4f14982f service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:19.739921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876236ce-fbd7-4940-b7a8-c43f4f14982f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:19.743828 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33803b14-f8ed-4b14-8528-a8b56233a784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:19.744642 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33803b14-f8ed-4b14-8528-a8b56233a784 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:19.745019 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33803b14-f8ed-4b14-8528-a8b56233a784 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:19.745328 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33803b14-f8ed-4b14-8528-a8b56233a784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:19.752239 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876236ce-fbd7-4940-b7a8-c43f4f14982f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:19.753813 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876236ce-fbd7-4940-b7a8-c43f4f14982f service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:19.763309 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33803b14-f8ed-4b14-8528-a8b56233a784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:19.765867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33803b14-f8ed-4b14-8528-a8b56233a784 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:19.770500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1201/2394] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:35:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:35:19.787890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1194/2395] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:35:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:35:20.587751 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876236ce-fbd7-4940-b7a8-c43f4f14982f service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:20.590543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-876236ce-fbd7-4940-b7a8-c43f4f14982f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:20.607180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876236ce-fbd7-4940-b7a8-c43f4f14982f service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:20.613681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f71dd-f4ea-44c3-881d-3d1e8cf07210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:20.615148 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f71dd-f4ea-44c3-881d-3d1e8cf07210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:20.615627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f71dd-f4ea-44c3-881d-3d1e8cf07210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:20.616105 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f71dd-f4ea-44c3-881d-3d1e8cf07210 admin admin] PATH_INFO: `/v3/projects/6224ef1e38214812af42576baceebee1` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:20.617518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21f71dd-f4ea-44c3-881d-3d1e8cf07210 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6224ef1e38214812af42576baceebee1)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:20.620572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21f71dd-f4ea-44c3-881d-3d1e8cf07210 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:20.761607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:20.771061 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c21f71dd-f4ea-44c3-881d-3d1e8cf07210 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6224ef1e38214812af42576baceebee1', 'request_id': 'req-c21f71dd-f4ea-44c3-881d-3d1e8cf07210'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:20.995332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c21f71dd-f4ea-44c3-881d-3d1e8cf07210 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6224ef1e38214812af42576baceebee1', 'request_id': 'req-c21f71dd-f4ea-44c3-881d-3d1e8cf07210'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:21.005254 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c21f71dd-f4ea-44c3-881d-3d1e8cf07210 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6224ef1e38214812af42576baceebee1', 'request_id': 'req-c21f71dd-f4ea-44c3-881d-3d1e8cf07210'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:21.023540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c21f71dd-f4ea-44c3-881d-3d1e8cf07210 admin admin] The token cache is being invalidate because project 6224ef1e38214812af42576baceebee1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:35:21.024471 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c21f71dd-f4ea-44c3-881d-3d1e8cf07210 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:21.025540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c21f71dd-f4ea-44c3-881d-3d1e8cf07210 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:35:21.031713 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1202/2396] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:35:20 2022] DELETE /identity/v3/projects/6224ef1e38214812af42576baceebee1 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:35:21.034041 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1ee90-1f41-4bae-840d-010ab86bd178 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:21.034523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1ee90-1f41-4bae-840d-010ab86bd178 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:21.035002 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1ee90-1f41-4bae-840d-010ab86bd178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:21.055919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-7eb1ee90-1f41-4bae-840d-010ab86bd178 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:21.057146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-7eb1ee90-1f41-4bae-840d-010ab86bd178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:21.128717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1195/2397] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:21.136757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd80b9-f166-4013-90c7-cd9bd13b74d3 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:21.137047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd80b9-f166-4013-90c7-cd9bd13b74d3 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:21.137327 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd80b9-f166-4013-90c7-cd9bd13b74d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:21.159069 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-fadd80b9-f166-4013-90c7-cd9bd13b74d3 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:21.165764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-fadd80b9-f166-4013-90c7-cd9bd13b74d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:21.171371 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1203/2398] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:21.175227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1ee90-1f41-4bae-840d-010ab86bd178 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:21.186841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1ee90-1f41-4bae-840d-010ab86bd178 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:21.193876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a06336-bf31-4568-92d3-86159f0d0cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:21.194587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a06336-bf31-4568-92d3-86159f0d0cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:21.194871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a06336-bf31-4568-92d3-86159f0d0cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:21.195191 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a06336-bf31-4568-92d3-86159f0d0cc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:21.195919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a06336-bf31-4568-92d3-86159f0d0cc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:35:21.196223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a06336-bf31-4568-92d3-86159f0d0cc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:21.197887 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a06336-bf31-4568-92d3-86159f0d0cc0 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:21.212361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1196/2399] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:35:21.217824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd80b9-f166-4013-90c7-cd9bd13b74d3 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:21.228565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd80b9-f166-4013-90c7-cd9bd13b74d3 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:21.233808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f50b2-b85c-4910-afac-526dfb8dfffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:21.234771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f50b2-b85c-4910-afac-526dfb8dfffa admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:21.235116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f50b2-b85c-4910-afac-526dfb8dfffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:21.235477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f50b2-b85c-4910-afac-526dfb8dfffa admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:21.236242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f50b2-b85c-4910-afac-526dfb8dfffa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:21.238298 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f50b2-b85c-4910-afac-526dfb8dfffa admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:21.366219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1204/2400] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:35:21 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:35:21.369557 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a06336-bf31-4568-92d3-86159f0d0cc0 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:21.386196 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a06336-bf31-4568-92d3-86159f0d0cc0 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:21.472381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5426c-58eb-4ef3-aabc-c38fae96fc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:21.473167 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5426c-58eb-4ef3-aabc-c38fae96fc1f admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:21.473517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5426c-58eb-4ef3-aabc-c38fae96fc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:21.473804 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5426c-58eb-4ef3-aabc-c38fae96fc1f admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:21.474600 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5426c-58eb-4ef3-aabc-c38fae96fc1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:21.477545 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5426c-58eb-4ef3-aabc-c38fae96fc1f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:21.626249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1197/2401] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:35:21 2022] POST /identity/v3/users => generated 292 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:35:21.630674 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f50b2-b85c-4910-afac-526dfb8dfffa admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:21.643379 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f50b2-b85c-4910-afac-526dfb8dfffa admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:21.649344 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53872325-cfad-4c23-8518-b05b8d5d8ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:21.650188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53872325-cfad-4c23-8518-b05b8d5d8ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:21.650506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53872325-cfad-4c23-8518-b05b8d5d8ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:21.650779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53872325-cfad-4c23-8518-b05b8d5d8ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:21.651509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53872325-cfad-4c23-8518-b05b8d5d8ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:21.653539 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53872325-cfad-4c23-8518-b05b8d5d8ae1 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:21.666196 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1205/2402] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:35:21.670255 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5426c-58eb-4ef3-aabc-c38fae96fc1f admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:21.683100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5426c-58eb-4ef3-aabc-c38fae96fc1f admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:21.689069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6da0da-2626-4642-8e0b-e75d471b0ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:21.689801 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6da0da-2626-4642-8e0b-e75d471b0ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:21.690241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6da0da-2626-4642-8e0b-e75d471b0ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:21.691142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6da0da-2626-4642-8e0b-e75d471b0ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:21.691440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6da0da-2626-4642-8e0b-e75d471b0ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:21.693078 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6da0da-2626-4642-8e0b-e75d471b0ab5 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:21.706538 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1198/2403] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:35:21.718561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53872325-cfad-4c23-8518-b05b8d5d8ae1 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:21.739121 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53872325-cfad-4c23-8518-b05b8d5d8ae1 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:21.745405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ada43-95b3-489d-9202-51a9ca0efd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:21.749039 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ada43-95b3-489d-9202-51a9ca0efd0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:21.749039 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ada43-95b3-489d-9202-51a9ca0efd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:21.749039 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ada43-95b3-489d-9202-51a9ca0efd0a admin admin] PATH_INFO: `/v3/projects/3e3b499bb69b4b449a2e2515d94094fb/users/9ea2abd1398f4fba93b3b7a5beb52ba0/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:21.768593 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ada43-95b3-489d-9202-51a9ca0efd0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e3b499bb69b4b449a2e2515d94094fb, user_id=9ea2abd1398f4fba93b3b7a5beb52ba0, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:21.771394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ada43-95b3-489d-9202-51a9ca0efd0a admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:21.942324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1206/2404] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:35:21 2022] PUT /identity/v3/projects/3e3b499bb69b4b449a2e2515d94094fb/users/9ea2abd1398f4fba93b3b7a5beb52ba0/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:35:21.943510 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186e866-bab6-49d8-a5db-4b7bab1ba868 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:21.944025 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186e866-bab6-49d8-a5db-4b7bab1ba868 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:21.944526 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186e866-bab6-49d8-a5db-4b7bab1ba868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:21.968088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-f186e866-bab6-49d8-a5db-4b7bab1ba868 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:21.969380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f186e866-bab6-49d8-a5db-4b7bab1ba868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:22.004355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:22.061891 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1199/2405] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:35:22.072305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ada43-95b3-489d-9202-51a9ca0efd0a admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:22.076132 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-431ada43-95b3-489d-9202-51a9ca0efd0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:22.159319 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ada43-95b3-489d-9202-51a9ca0efd0a admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:22.164860 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254c291-de5b-4c36-bfc2-30dda0ab4b07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:22.166083 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254c291-de5b-4c36-bfc2-30dda0ab4b07 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:22.166896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254c291-de5b-4c36-bfc2-30dda0ab4b07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:22.167221 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254c291-de5b-4c36-bfc2-30dda0ab4b07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:22.189902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254c291-de5b-4c36-bfc2-30dda0ab4b07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:22.193483 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254c291-de5b-4c36-bfc2-30dda0ab4b07 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:22.212086 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1207/2406] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:35:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:35:22.341423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0591e-96e2-4455-b67e-8a7137c95022 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:22.341718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0591e-96e2-4455-b67e-8a7137c95022 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:22.342202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0591e-96e2-4455-b67e-8a7137c95022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:22.364632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-a9d0591e-96e2-4455-b67e-8a7137c95022 None None] MFA Rules not processed for user `4e75a82cc7e94702ba0080fc65d6eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:22.366687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a9d0591e-96e2-4455-b67e-8a7137c95022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:22.457289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1200/2407] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Jul 20 15:35:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:22.474286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254c291-de5b-4c36-bfc2-30dda0ab4b07 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:22.476376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1254c291-de5b-4c36-bfc2-30dda0ab4b07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:22.591898 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254c291-de5b-4c36-bfc2-30dda0ab4b07 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:22.599161 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fffb7-84e0-4e7c-9371-b758ced21922 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0d719c0eb04a3b8a228c6ce70fdf98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:22.599853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fffb7-84e0-4e7c-9371-b758ced21922 service swift] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:22.600138 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fffb7-84e0-4e7c-9371-b758ced21922 service swift] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:22.600733 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fffb7-84e0-4e7c-9371-b758ced21922 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:22.612456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:22.616112 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fffb7-84e0-4e7c-9371-b758ced21922 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:22.617790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fffb7-84e0-4e7c-9371-b758ced21922 service swift] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:22.628360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1208/2408] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Jul 20 15:35:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:35:26.619368 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56aaef-88c2-4c7f-943f-a280d21f87b4 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:26.620265 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56aaef-88c2-4c7f-943f-a280d21f87b4 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:26.620573 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56aaef-88c2-4c7f-943f-a280d21f87b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:26.680242 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-8b56aaef-88c2-4c7f-943f-a280d21f87b4 None None] MFA Rules not processed for user `9ea2abd1398f4fba93b3b7a5beb52ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:26.682124 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8b56aaef-88c2-4c7f-943f-a280d21f87b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:26.889573 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1201/2409] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:26.902956 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fffb7-84e0-4e7c-9371-b758ced21922 service swift] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:26.909670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-569fffb7-84e0-4e7c-9371-b758ced21922 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:27.039321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fffb7-84e0-4e7c-9371-b758ced21922 service swift] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:27.064680 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e6c6b-cfc3-4802-bd3c-de03eecb2849 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:27.065624 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e6c6b-cfc3-4802-bd3c-de03eecb2849 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:27.065925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e6c6b-cfc3-4802-bd3c-de03eecb2849 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:27.066314 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e6c6b-cfc3-4802-bd3c-de03eecb2849 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:27.087417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e6c6b-cfc3-4802-bd3c-de03eecb2849 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:27.093977 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e6c6b-cfc3-4802-bd3c-de03eecb2849 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:27.121806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1209/2410] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:35:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:35:29.361717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cc7d3-ddbc-4c6d-b682-8725ccb0f1f7 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:29.361717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cc7d3-ddbc-4c6d-b682-8725ccb0f1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:29.361717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cc7d3-ddbc-4c6d-b682-8725ccb0f1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:29.365935 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-e64cc7d3-ddbc-4c6d-b682-8725ccb0f1f7 None None] MFA Rules not processed for user `9ea2abd1398f4fba93b3b7a5beb52ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:29.385114 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e64cc7d3-ddbc-4c6d-b682-8725ccb0f1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:29.422666 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1202/2411] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:31.381702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b60ed-78bc-45df-bc40-ad63ce7a875b None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:31.381702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b60ed-78bc-45df-bc40-ad63ce7a875b None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:31.381702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b60ed-78bc-45df-bc40-ad63ce7a875b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:31.436556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-e82b60ed-78bc-45df-bc40-ad63ce7a875b None None] MFA Rules not processed for user `ec2e023235a343dd81754211ee102f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:31.439267 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e82b60ed-78bc-45df-bc40-ad63ce7a875b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:31.596983 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1210/2412] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:31.597323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cc7d3-ddbc-4c6d-b682-8725ccb0f1f7 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:31.599855 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e64cc7d3-ddbc-4c6d-b682-8725ccb0f1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:31.613805 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cc7d3-ddbc-4c6d-b682-8725ccb0f1f7 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:31.640973 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b2676-8ba6-4cb6-b377-48caa65b6662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:31.642028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b2676-8ba6-4cb6-b377-48caa65b6662 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:31.642544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b2676-8ba6-4cb6-b377-48caa65b6662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:31.642979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b2676-8ba6-4cb6-b377-48caa65b6662 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:31.644154 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b2676-8ba6-4cb6-b377-48caa65b6662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:31.646339 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b2676-8ba6-4cb6-b377-48caa65b6662 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:31.777324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b60ed-78bc-45df-bc40-ad63ce7a875b None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:31.782965 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1203/2413] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:35:31 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:35:31.790291 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b60ed-78bc-45df-bc40-ad63ce7a875b None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:31.932803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d13621-c69e-4c57-8e30-cd501f67b519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:31.950584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d13621-c69e-4c57-8e30-cd501f67b519 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:31.950584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d13621-c69e-4c57-8e30-cd501f67b519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:31.950584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d13621-c69e-4c57-8e30-cd501f67b519 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:31.950584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d13621-c69e-4c57-8e30-cd501f67b519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:31.950584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d13621-c69e-4c57-8e30-cd501f67b519 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:32.254317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1211/2414] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:35:31 2022] POST /identity/v3/users => generated 293 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:35:32.255779 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b2676-8ba6-4cb6-b377-48caa65b6662 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:32.273420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b2676-8ba6-4cb6-b377-48caa65b6662 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:32.285556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ea3b8-f809-464f-94ea-e9d141442cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:32.286635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ea3b8-f809-464f-94ea-e9d141442cae admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:32.287125 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ea3b8-f809-464f-94ea-e9d141442cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:32.287617 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ea3b8-f809-464f-94ea-e9d141442cae admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:32.288555 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ea3b8-f809-464f-94ea-e9d141442cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:32.290425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ea3b8-f809-464f-94ea-e9d141442cae admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:32.312868 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1204/2415] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:35:32.317653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d13621-c69e-4c57-8e30-cd501f67b519 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:32.330497 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d13621-c69e-4c57-8e30-cd501f67b519 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:32.360374 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503c0da-776a-4343-9ce7-687d0384f1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:32.361782 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503c0da-776a-4343-9ce7-687d0384f1a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:32.362076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503c0da-776a-4343-9ce7-687d0384f1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:32.362374 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503c0da-776a-4343-9ce7-687d0384f1a0 admin admin] PATH_INFO: `/v3/projects/24edc8866f044562ad23afe2815af7b0/users/a98af3ac148a4c72830f13182195fd13/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:32.399819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503c0da-776a-4343-9ce7-687d0384f1a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24edc8866f044562ad23afe2815af7b0, user_id=a98af3ac148a4c72830f13182195fd13, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:32.399819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503c0da-776a-4343-9ce7-687d0384f1a0 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:32.399819 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:32.515482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1212/2416] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:35:32 2022] PUT /identity/v3/projects/24edc8866f044562ad23afe2815af7b0/users/a98af3ac148a4c72830f13182195fd13/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:35:32.516938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ea3b8-f809-464f-94ea-e9d141442cae admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:32.532089 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ea3b8-f809-464f-94ea-e9d141442cae admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:32.571940 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:32.657969 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6ec52-877e-48bd-9261-4274b0a3ff08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:32.662431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6ec52-877e-48bd-9261-4274b0a3ff08 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:32.662431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6ec52-877e-48bd-9261-4274b0a3ff08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:32.662431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6ec52-877e-48bd-9261-4274b0a3ff08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:32.662431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6ec52-877e-48bd-9261-4274b0a3ff08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:32.670994 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6ec52-877e-48bd-9261-4274b0a3ff08 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:32.692936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1205/2417] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:35:32.697804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503c0da-776a-4343-9ce7-687d0384f1a0 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:32.729207 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503c0da-776a-4343-9ce7-687d0384f1a0 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:32.751023 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe334f-ec61-4ae5-9275-944511c1b4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:32.752230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe334f-ec61-4ae5-9275-944511c1b4a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:32.752726 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe334f-ec61-4ae5-9275-944511c1b4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:32.753204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe334f-ec61-4ae5-9275-944511c1b4a1 admin admin] PATH_INFO: `/v3/projects/24edc8866f044562ad23afe2815af7b0/users/a98af3ac148a4c72830f13182195fd13/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:32.755813 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe334f-ec61-4ae5-9275-944511c1b4a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24edc8866f044562ad23afe2815af7b0, user_id=a98af3ac148a4c72830f13182195fd13, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:32.759070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe334f-ec61-4ae5-9275-944511c1b4a1 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:32.797963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1213/2418] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:35:32 2022] PUT /identity/v3/projects/24edc8866f044562ad23afe2815af7b0/users/a98af3ac148a4c72830f13182195fd13/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:35:41.109256 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53f1ba-3a29-4ba0-a703-b8846f472240 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:41.110081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53f1ba-3a29-4ba0-a703-b8846f472240 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:41.110458 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53f1ba-3a29-4ba0-a703-b8846f472240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:41.141209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-7f53f1ba-3a29-4ba0-a703-b8846f472240 None None] MFA Rules not processed for user `a98af3ac148a4c72830f13182195fd13`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:41.142646 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-7f53f1ba-3a29-4ba0-a703-b8846f472240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:41.216237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1206/2419] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:35:41.231485 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe334f-ec61-4ae5-9275-944511c1b4a1 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:41.232220 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2fbe334f-ec61-4ae5-9275-944511c1b4a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:41.325936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe334f-ec61-4ae5-9275-944511c1b4a1 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:41.332327 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d7895-9b76-468e-a599-4b49494a5963 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:41.334584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d7895-9b76-468e-a599-4b49494a5963 service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:41.334584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d7895-9b76-468e-a599-4b49494a5963 service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:41.334584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d7895-9b76-468e-a599-4b49494a5963 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:41.405606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d7895-9b76-468e-a599-4b49494a5963 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:41.407502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d7895-9b76-468e-a599-4b49494a5963 service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:41.418321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1214/2420] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:35:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:35:45.602501 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb685d87-8f93-4118-9ab0-7d7a34207557 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:45.602978 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb685d87-8f93-4118-9ab0-7d7a34207557 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:45.603086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb685d87-8f93-4118-9ab0-7d7a34207557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:45.621623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-fb685d87-8f93-4118-9ab0-7d7a34207557 None None] MFA Rules not processed for user `ec2e023235a343dd81754211ee102f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:45.622857 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-fb685d87-8f93-4118-9ab0-7d7a34207557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:45.629705 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1207/2421] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:35:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:47.942839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fabae3-9c37-444a-8115-5834121adb28 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:47.943363 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fabae3-9c37-444a-8115-5834121adb28 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:47.943459 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fabae3-9c37-444a-8115-5834121adb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:47.966014 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-d0fabae3-9c37-444a-8115-5834121adb28 None None] MFA Rules not processed for user `4e75a82cc7e94702ba0080fc65d6eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:35:47.967267 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d0fabae3-9c37-444a-8115-5834121adb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:48.025860 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1215/2422] 10.222.0.78 () {58 vars in 967 bytes} [Wed Jul 20 15:35:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:35:48.048606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb685d87-8f93-4118-9ab0-7d7a34207557 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:48.169425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb685d87-8f93-4118-9ab0-7d7a34207557 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:48.176674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f266c-46e8-4164-a9d5-957b0294be40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0d719c0eb04a3b8a228c6ce70fdf98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:48.177166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f266c-46e8-4164-a9d5-957b0294be40 service swift] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:48.177447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f266c-46e8-4164-a9d5-957b0294be40 service swift] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:48.177721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f266c-46e8-4164-a9d5-957b0294be40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:48.188416 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f266c-46e8-4164-a9d5-957b0294be40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:48.190316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f266c-46e8-4164-a9d5-957b0294be40 service swift] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:48.204997 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1208/2423] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Jul 20 15:35:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:35:53.466138 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fabae3-9c37-444a-8115-5834121adb28 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:35:53.507417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:35:53.536936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fabae3-9c37-444a-8115-5834121adb28 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:35:53.540367 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8d4ce-c9b2-435a-b19e-f92e20650900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:35:53.541415 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8d4ce-c9b2-435a-b19e-f92e20650900 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:35:53.541589 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8d4ce-c9b2-435a-b19e-f92e20650900 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:35:53.542431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8d4ce-c9b2-435a-b19e-f92e20650900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:35:53.544633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-4cc8d4ce-c9b2-435a-b19e-f92e20650900 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:35:53.619680 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8d4ce-c9b2-435a-b19e-f92e20650900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:35:53.621718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8d4ce-c9b2-435a-b19e-f92e20650900 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:35:53.631769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1216/2424] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:35:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:35:58.199152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=88 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:03.755516 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=40 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:09.803267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702db0f-6d70-49ed-95eb-1ce751fc7b91 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:09.803956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702db0f-6d70-49ed-95eb-1ce751fc7b91 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:09.803956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702db0f-6d70-49ed-95eb-1ce751fc7b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:09.821416 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-6702db0f-6d70-49ed-95eb-1ce751fc7b91 None None] MFA Rules not processed for user `ebbcfe6341a3486d93e12c09f83954dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:36:09.826337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-6702db0f-6d70-49ed-95eb-1ce751fc7b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:09.887672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1209/2425] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:36:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:36:19.879286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=25 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:37.795459 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8d4ce-c9b2-435a-b19e-f92e20650900 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:37.810136 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8d4ce-c9b2-435a-b19e-f92e20650900 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:37.816113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a57664-07d4-4c4f-9c5e-3fad5eed76d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:37.817231 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a57664-07d4-4c4f-9c5e-3fad5eed76d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:37.817641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a57664-07d4-4c4f-9c5e-3fad5eed76d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:37.818065 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a57664-07d4-4c4f-9c5e-3fad5eed76d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:37.830883 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a57664-07d4-4c4f-9c5e-3fad5eed76d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:37.832325 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a57664-07d4-4c4f-9c5e-3fad5eed76d6 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:37.850857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1217/2426] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:39.314481 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702db0f-6d70-49ed-95eb-1ce751fc7b91 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:39.325259 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702db0f-6d70-49ed-95eb-1ce751fc7b91 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:39.331470 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa329d7-a4a6-471f-9c0f-9e66dba2b550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:39.336210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa329d7-a4a6-471f-9c0f-9e66dba2b550 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:39.336210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa329d7-a4a6-471f-9c0f-9e66dba2b550 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:39.336210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa329d7-a4a6-471f-9c0f-9e66dba2b550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:39.337977 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-baa329d7-a4a6-471f-9c0f-9e66dba2b550 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:39.353188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa329d7-a4a6-471f-9c0f-9e66dba2b550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:39.354602 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa329d7-a4a6-471f-9c0f-9e66dba2b550 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:39.366617 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1210/2427] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:36:44.355912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a57664-07d4-4c4f-9c5e-3fad5eed76d6 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:44.365452 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a57664-07d4-4c4f-9c5e-3fad5eed76d6 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:44.369606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc40442-7e39-4184-87eb-c2e377d77fb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:44.370247 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc40442-7e39-4184-87eb-c2e377d77fb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:44.370562 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc40442-7e39-4184-87eb-c2e377d77fb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:44.370831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc40442-7e39-4184-87eb-c2e377d77fb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:44.374212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-bdc40442-7e39-4184-87eb-c2e377d77fb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:44.391730 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc40442-7e39-4184-87eb-c2e377d77fb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:44.395361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc40442-7e39-4184-87eb-c2e377d77fb6 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:44.405594 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1218/2428] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:49.361525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=6 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:54.400211 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=12 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:36:56.250355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa329d7-a4a6-471f-9c0f-9e66dba2b550 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:56.308293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa329d7-a4a6-471f-9c0f-9e66dba2b550 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:56.312020 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fc0b5-0fe1-43ef-844c-0c618964a2c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:56.312664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fc0b5-0fe1-43ef-844c-0c618964a2c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:56.312943 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fc0b5-0fe1-43ef-844c-0c618964a2c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:56.313214 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fc0b5-0fe1-43ef-844c-0c618964a2c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:56.315060 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e96fc0b5-0fe1-43ef-844c-0c618964a2c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:56.386027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96fc0b5-0fe1-43ef-844c-0c618964a2c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:56.387593 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96fc0b5-0fe1-43ef-844c-0c618964a2c6 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:56.400366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1211/2429] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:36:57.688445 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc40442-7e39-4184-87eb-c2e377d77fb6 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:36:57.690178 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-bdc40442-7e39-4184-87eb-c2e377d77fb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:57.709719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc40442-7e39-4184-87eb-c2e377d77fb6 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:36:57.715829 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ed7ff-8715-45ad-b212-ac1e083b57d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:36:57.717077 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ed7ff-8715-45ad-b212-ac1e083b57d8 service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:36:57.717490 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ed7ff-8715-45ad-b212-ac1e083b57d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:36:57.717839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ed7ff-8715-45ad-b212-ac1e083b57d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:36:57.719827 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-072ed7ff-8715-45ad-b212-ac1e083b57d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:36:57.789614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ed7ff-8715-45ad-b212-ac1e083b57d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:36:57.791327 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ed7ff-8715-45ad-b212-ac1e083b57d8 service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:36:57.801497 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1219/2430] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:36:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:37:03.045485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901513e0-dfef-4d43-8b85-931ff1edbb08 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:03.046004 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901513e0-dfef-4d43-8b85-931ff1edbb08 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:03.046097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901513e0-dfef-4d43-8b85-931ff1edbb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:03.066455 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-901513e0-dfef-4d43-8b85-931ff1edbb08 None None] MFA Rules not processed for user `9ea2abd1398f4fba93b3b7a5beb52ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:03.067719 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-901513e0-dfef-4d43-8b85-931ff1edbb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:03.123592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1212/2431] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:37:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:03.447738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ed7ff-8715-45ad-b212-ac1e083b57d8 service nova] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:03.460071 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ed7ff-8715-45ad-b212-ac1e083b57d8 service nova] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:03.463693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd61e4-ef02-4498-978b-078ccb10e17e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:03.464334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd61e4-ef02-4498-978b-078ccb10e17e service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:03.464623 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd61e4-ef02-4498-978b-078ccb10e17e service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:03.464901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd61e4-ef02-4498-978b-078ccb10e17e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:03.467319 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6dcd61e4-ef02-4498-978b-078ccb10e17e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:03.476933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd61e4-ef02-4498-978b-078ccb10e17e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:03.478559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd61e4-ef02-4498-978b-078ccb10e17e service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:03.487340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1220/2432] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:03.910587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901513e0-dfef-4d43-8b85-931ff1edbb08 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:03.912153 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-901513e0-dfef-4d43-8b85-931ff1edbb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:03.991445 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:03.997773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901513e0-dfef-4d43-8b85-931ff1edbb08 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:04.003430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26552f16-5752-441a-a96e-ff8682342205 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2192547babde4449a46f9cb37ed64524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:04.003430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26552f16-5752-441a-a96e-ff8682342205 service placement] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:04.003670 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26552f16-5752-441a-a96e-ff8682342205 service placement] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:04.003888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26552f16-5752-441a-a96e-ff8682342205 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:04.006527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-26552f16-5752-441a-a96e-ff8682342205 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:04.018446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26552f16-5752-441a-a96e-ff8682342205 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:04.020009 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26552f16-5752-441a-a96e-ff8682342205 service placement] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:04.030225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1213/2433] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:07.962531 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd61e4-ef02-4498-978b-078ccb10e17e service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:07.964207 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6dcd61e4-ef02-4498-978b-078ccb10e17e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:07.975963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd61e4-ef02-4498-978b-078ccb10e17e service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:07.980464 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7fcde-460d-43fd-ac1d-d8a7a1023f12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:07.981159 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7fcde-460d-43fd-ac1d-d8a7a1023f12 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:07.981971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7fcde-460d-43fd-ac1d-d8a7a1023f12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:07.982316 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7fcde-460d-43fd-ac1d-d8a7a1023f12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:07.984319 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-13a7fcde-460d-43fd-ac1d-d8a7a1023f12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:07.995902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7fcde-460d-43fd-ac1d-d8a7a1023f12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:07.998024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7fcde-460d-43fd-ac1d-d8a7a1023f12 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:08.008023 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1221/2434] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:13.051529 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1489ec-0965-4f49-b354-b277c1e318e7 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:13.051980 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1489ec-0965-4f49-b354-b277c1e318e7 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:13.052076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1489ec-0965-4f49-b354-b277c1e318e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:13.070701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-ba1489ec-0965-4f49-b354-b277c1e318e7 None None] MFA Rules not processed for user `6498ef5c517c4fa4abed90e430936ded`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:13.071899 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ba1489ec-0965-4f49-b354-b277c1e318e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:13.079632 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1214/2435] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:37:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:37:14.064229 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145955b-3680-498f-8b24-f3ed2504f876 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:14.064954 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145955b-3680-498f-8b24-f3ed2504f876 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:14.064954 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145955b-3680-498f-8b24-f3ed2504f876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:14.084942 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-8145955b-3680-498f-8b24-f3ed2504f876 None None] MFA Rules not processed for user `8e75c180bae74be0b2ccad079f92d32b`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:14.086220 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8145955b-3680-498f-8b24-f3ed2504f876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:14.098506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1222/2436] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:37:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:14.467107 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1489ec-0965-4f49-b354-b277c1e318e7 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:14.477077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1489ec-0965-4f49-b354-b277c1e318e7 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:14.480989 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29033d2e-0c40-4985-8408-cbaaaaa2cedd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:14.482166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29033d2e-0c40-4985-8408-cbaaaaa2cedd service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:14.482388 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29033d2e-0c40-4985-8408-cbaaaaa2cedd service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:14.482621 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29033d2e-0c40-4985-8408-cbaaaaa2cedd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:14.484754 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-29033d2e-0c40-4985-8408-cbaaaaa2cedd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:14.496903 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29033d2e-0c40-4985-8408-cbaaaaa2cedd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:14.498596 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29033d2e-0c40-4985-8408-cbaaaaa2cedd service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:14.508206 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1215/2437] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:14.918525 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145955b-3680-498f-8b24-f3ed2504f876 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:14.920046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8145955b-3680-498f-8b24-f3ed2504f876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:14.932334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145955b-3680-498f-8b24-f3ed2504f876 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:14.936800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fefe1-9d68-4909-bbcb-637de5368908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:14.937524 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fefe1-9d68-4909-bbcb-637de5368908 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:14.941936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fefe1-9d68-4909-bbcb-637de5368908 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:14.941936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fefe1-9d68-4909-bbcb-637de5368908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:14.941936 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-744fefe1-9d68-4909-bbcb-637de5368908 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:14.952604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fefe1-9d68-4909-bbcb-637de5368908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:14.954322 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fefe1-9d68-4909-bbcb-637de5368908 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:14.965047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1223/2438] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:24.055350 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29033d2e-0c40-4985-8408-cbaaaaa2cedd service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:24.073882 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29033d2e-0c40-4985-8408-cbaaaaa2cedd service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:24.085109 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf8bf7-f6dd-4ade-bae2-c7158898cec1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:24.085743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf8bf7-f6dd-4ade-bae2-c7158898cec1 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:24.086156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf8bf7-f6dd-4ade-bae2-c7158898cec1 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:24.086423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf8bf7-f6dd-4ade-bae2-c7158898cec1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:24.088663 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-06cf8bf7-f6dd-4ade-bae2-c7158898cec1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:24.107485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf8bf7-f6dd-4ade-bae2-c7158898cec1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:24.109116 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf8bf7-f6dd-4ade-bae2-c7158898cec1 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:24.131602 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1216/2439] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:24.960077 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=49 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:25.019714 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fefe1-9d68-4909-bbcb-637de5368908 service cinder] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:25.037552 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fefe1-9d68-4909-bbcb-637de5368908 service cinder] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:25.046817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd004a13-d8ee-4adc-a2ef-9296887c0141 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:25.048264 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd004a13-d8ee-4adc-a2ef-9296887c0141 service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:25.049187 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd004a13-d8ee-4adc-a2ef-9296887c0141 service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:25.049714 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd004a13-d8ee-4adc-a2ef-9296887c0141 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:25.051907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-bd004a13-d8ee-4adc-a2ef-9296887c0141 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:25.067710 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd004a13-d8ee-4adc-a2ef-9296887c0141 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:25.071226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd004a13-d8ee-4adc-a2ef-9296887c0141 service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:25.086988 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1224/2440] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:34.122194 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=20 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:35.433617 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=6 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:49.590586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf8bf7-f6dd-4ade-bae2-c7158898cec1 service nova] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:49.600154 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf8bf7-f6dd-4ade-bae2-c7158898cec1 service nova] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:49.603728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b7d0f-0d6b-4e4d-91a1-f41ecc4e2f58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:49.604511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b7d0f-0d6b-4e4d-91a1-f41ecc4e2f58 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:49.604789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b7d0f-0d6b-4e4d-91a1-f41ecc4e2f58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:49.605063 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b7d0f-0d6b-4e4d-91a1-f41ecc4e2f58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:49.606881 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8a3b7d0f-0d6b-4e4d-91a1-f41ecc4e2f58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:49.676032 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b7d0f-0d6b-4e4d-91a1-f41ecc4e2f58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:49.677460 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b7d0f-0d6b-4e4d-91a1-f41ecc4e2f58 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:49.687803 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1217/2441] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:37:54.825706 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd004a13-d8ee-4adc-a2ef-9296887c0141 service nova] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:54.827736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-bd004a13-d8ee-4adc-a2ef-9296887c0141 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:54.848010 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd004a13-d8ee-4adc-a2ef-9296887c0141 service nova] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:54.852655 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f7f2a-87ca-46d0-9dca-5ee42c827a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:54.859465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f7f2a-87ca-46d0-9dca-5ee42c827a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:54.859465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f7f2a-87ca-46d0-9dca-5ee42c827a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:54.859465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f7f2a-87ca-46d0-9dca-5ee42c827a10 admin admin] PATH_INFO: `/v3/users/ebbcfe6341a3486d93e12c09f83954dc` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:54.859465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f7f2a-87ca-46d0-9dca-5ee42c827a10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebbcfe6341a3486d93e12c09f83954dc)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:54.859465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f7f2a-87ca-46d0-9dca-5ee42c827a10 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:55.132898 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-6c6f7f2a-87ca-46d0-9dca-5ee42c827a10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebbcfe6341a3486d93e12c09f83954dc'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:55.219932 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-6c6f7f2a-87ca-46d0-9dca-5ee42c827a10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebbcfe6341a3486d93e12c09f83954dc'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:55.227405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-6c6f7f2a-87ca-46d0-9dca-5ee42c827a10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebbcfe6341a3486d93e12c09f83954dc'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:55.230229 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-6c6f7f2a-87ca-46d0-9dca-5ee42c827a10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebbcfe6341a3486d93e12c09f83954dc'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:55.239937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-6c6f7f2a-87ca-46d0-9dca-5ee42c827a10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebbcfe6341a3486d93e12c09f83954dc'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:55.246373 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1225/2442] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:37:54 2022] DELETE /identity/v3/users/ebbcfe6341a3486d93e12c09f83954dc => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:55.514220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b7d0f-0d6b-4e4d-91a1-f41ecc4e2f58 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:55.520741 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b7d0f-0d6b-4e4d-91a1-f41ecc4e2f58 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:55.579771 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bfd61-267d-41b3-b45c-3e0c7e099e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:55.580664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bfd61-267d-41b3-b45c-3e0c7e099e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:55.580948 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bfd61-267d-41b3-b45c-3e0c7e099e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:55.581225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bfd61-267d-41b3-b45c-3e0c7e099e3a admin admin] PATH_INFO: `/v3/projects/61872ef687074f5791a2cc46732bebdf` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:55.582455 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613bfd61-267d-41b3-b45c-3e0c7e099e3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=61872ef687074f5791a2cc46732bebdf)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:55.584602 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613bfd61-267d-41b3-b45c-3e0c7e099e3a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:55.720162 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-613bfd61-267d-41b3-b45c-3e0c7e099e3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61872ef687074f5791a2cc46732bebdf', 'request_id': 'req-613bfd61-267d-41b3-b45c-3e0c7e099e3a'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:55.798620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-613bfd61-267d-41b3-b45c-3e0c7e099e3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61872ef687074f5791a2cc46732bebdf', 'request_id': 'req-613bfd61-267d-41b3-b45c-3e0c7e099e3a'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:55.805419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-613bfd61-267d-41b3-b45c-3e0c7e099e3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61872ef687074f5791a2cc46732bebdf', 'request_id': 'req-613bfd61-267d-41b3-b45c-3e0c7e099e3a'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:55.816608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-613bfd61-267d-41b3-b45c-3e0c7e099e3a admin admin] The token cache is being invalidate because project 61872ef687074f5791a2cc46732bebdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:37:55.816899 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-613bfd61-267d-41b3-b45c-3e0c7e099e3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:55.817542 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-613bfd61-267d-41b3-b45c-3e0c7e099e3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:37:55.819109 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1218/2443] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:37:55 2022] DELETE /identity/v3/projects/61872ef687074f5791a2cc46732bebdf => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:55.824097 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36d43a-6f6f-4d71-9642-35e2e53f219c None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:55.824384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36d43a-6f6f-4d71-9642-35e2e53f219c None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:55.824661 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36d43a-6f6f-4d71-9642-35e2e53f219c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:55.843201 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-cb36d43a-6f6f-4d71-9642-35e2e53f219c None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:55.844486 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-cb36d43a-6f6f-4d71-9642-35e2e53f219c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:55.903730 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1226/2444] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:55.911723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94359998-0bf4-4fc5-a989-fa0aa835cda5 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:55.915555 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94359998-0bf4-4fc5-a989-fa0aa835cda5 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:55.915555 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94359998-0bf4-4fc5-a989-fa0aa835cda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:55.931384 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-94359998-0bf4-4fc5-a989-fa0aa835cda5 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:55.932710 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-94359998-0bf4-4fc5-a989-fa0aa835cda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:55.941215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1219/2445] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:55.945299 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36d43a-6f6f-4d71-9642-35e2e53f219c None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:55.955181 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36d43a-6f6f-4d71-9642-35e2e53f219c None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:55.960095 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238a21b-80bd-43d2-99da-1dda0771e7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:55.960768 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238a21b-80bd-43d2-99da-1dda0771e7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:55.961047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238a21b-80bd-43d2-99da-1dda0771e7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:55.961473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238a21b-80bd-43d2-99da-1dda0771e7f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:55.962088 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5238a21b-80bd-43d2-99da-1dda0771e7f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:37:55.962434 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5238a21b-80bd-43d2-99da-1dda0771e7f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:55.964023 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5238a21b-80bd-43d2-99da-1dda0771e7f6 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:55.981404 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1227/2446] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:37:55.983118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94359998-0bf4-4fc5-a989-fa0aa835cda5 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:55.992669 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94359998-0bf4-4fc5-a989-fa0aa835cda5 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:55.997984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd7a25-5c96-42c8-9a4f-43a3f3a92326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:55.998667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd7a25-5c96-42c8-9a4f-43a3f3a92326 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:55.998950 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd7a25-5c96-42c8-9a4f-43a3f3a92326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:55.999226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd7a25-5c96-42c8-9a4f-43a3f3a92326 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:55.999918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd7a25-5c96-42c8-9a4f-43a3f3a92326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:56.001857 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd7a25-5c96-42c8-9a4f-43a3f3a92326 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:56.017308 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:56.153935 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1220/2447] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:37:55 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:37:56.155051 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238a21b-80bd-43d2-99da-1dda0771e7f6 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:56.164979 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238a21b-80bd-43d2-99da-1dda0771e7f6 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:56.220458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99ff37-3ff0-45ba-81cd-80e14c82ce48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:56.221129 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99ff37-3ff0-45ba-81cd-80e14c82ce48 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:56.221410 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99ff37-3ff0-45ba-81cd-80e14c82ce48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:56.221685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99ff37-3ff0-45ba-81cd-80e14c82ce48 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:56.222398 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99ff37-3ff0-45ba-81cd-80e14c82ce48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:56.224317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99ff37-3ff0-45ba-81cd-80e14c82ce48 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:56.231394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:37:56.376367 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1228/2448] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:37:56 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:37:56.380504 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd7a25-5c96-42c8-9a4f-43a3f3a92326 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:56.391350 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd7a25-5c96-42c8-9a4f-43a3f3a92326 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:56.396028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e38e4-bda5-4862-95fb-10f135e2e54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:56.396688 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e38e4-bda5-4862-95fb-10f135e2e54a admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:56.396972 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e38e4-bda5-4862-95fb-10f135e2e54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:56.397248 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e38e4-bda5-4862-95fb-10f135e2e54a admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:56.397981 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e38e4-bda5-4862-95fb-10f135e2e54a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:56.399646 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e38e4-bda5-4862-95fb-10f135e2e54a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:56.410718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1221/2449] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:56.414781 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99ff37-3ff0-45ba-81cd-80e14c82ce48 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:56.424907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99ff37-3ff0-45ba-81cd-80e14c82ce48 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:56.429475 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2dd6ec-64fd-4f4f-be33-f1210d633418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:56.430130 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2dd6ec-64fd-4f4f-be33-f1210d633418 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:56.430441 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2dd6ec-64fd-4f4f-be33-f1210d633418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:56.430719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2dd6ec-64fd-4f4f-be33-f1210d633418 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:56.431465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2dd6ec-64fd-4f4f-be33-f1210d633418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:56.433205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2dd6ec-64fd-4f4f-be33-f1210d633418 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:56.444319 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1229/2450] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:37:56.449006 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e38e4-bda5-4862-95fb-10f135e2e54a admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:56.459160 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e38e4-bda5-4862-95fb-10f135e2e54a admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:56.468524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd5def-8888-408e-9b80-5822e1e26faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:56.468524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd5def-8888-408e-9b80-5822e1e26faf admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:56.468524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd5def-8888-408e-9b80-5822e1e26faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:56.468524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd5def-8888-408e-9b80-5822e1e26faf admin admin] PATH_INFO: `/v3/projects/44dc2a5c754d42d1862b331afef2183f/users/0235b8fe92554131ba30e1d18f0f2d86/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:56.481054 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd5def-8888-408e-9b80-5822e1e26faf admin admin] RBAC: Authorizing `identity:create_grant(project_id=44dc2a5c754d42d1862b331afef2183f, user_id=0235b8fe92554131ba30e1d18f0f2d86, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:56.483687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd5def-8888-408e-9b80-5822e1e26faf admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:56.593803 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1222/2451] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:37:56 2022] PUT /identity/v3/projects/44dc2a5c754d42d1862b331afef2183f/users/0235b8fe92554131ba30e1d18f0f2d86/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:37:56.609906 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36665bb-a7e2-44e0-9068-257eae76499a None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:56.610212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36665bb-a7e2-44e0-9068-257eae76499a None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:56.610518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36665bb-a7e2-44e0-9068-257eae76499a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:56.630215 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-f36665bb-a7e2-44e0-9068-257eae76499a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:37:56.631381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f36665bb-a7e2-44e0-9068-257eae76499a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:56.810652 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1230/2452] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:37:56.820383 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd5def-8888-408e-9b80-5822e1e26faf admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:37:56.822091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-cfdd5def-8888-408e-9b80-5822e1e26faf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:37:56.880886 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd5def-8888-408e-9b80-5822e1e26faf admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:37:56.884361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece60f7a-7dc5-40c7-8172-d29c891ab0e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:37:56.885006 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece60f7a-7dc5-40c7-8172-d29c891ab0e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:37:56.885293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece60f7a-7dc5-40c7-8172-d29c891ab0e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:37:56.885569 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece60f7a-7dc5-40c7-8172-d29c891ab0e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:37:56.894829 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece60f7a-7dc5-40c7-8172-d29c891ab0e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:37:56.897942 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece60f7a-7dc5-40c7-8172-d29c891ab0e6 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:37:56.906159 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1223/2453] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:37:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:38:03.696601 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82727712-3c3b-4291-877f-3398130fd45b None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:03.697080 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82727712-3c3b-4291-877f-3398130fd45b None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:03.697188 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82727712-3c3b-4291-877f-3398130fd45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:03.727415 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-82727712-3c3b-4291-877f-3398130fd45b None None] MFA Rules not processed for user `0235b8fe92554131ba30e1d18f0f2d86`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:03.728840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-82727712-3c3b-4291-877f-3398130fd45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:03.799298 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1231/2454] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:38:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:03.814549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece60f7a-7dc5-40c7-8172-d29c891ab0e6 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:03.816225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ece60f7a-7dc5-40c7-8172-d29c891ab0e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:03.897744 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece60f7a-7dc5-40c7-8172-d29c891ab0e6 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:03.903878 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fdcbec-b4a5-4578-9c5c-b2eb036a9c8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:03.904981 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdcbec-b4a5-4578-9c5c-b2eb036a9c8c service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:03.905316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdcbec-b4a5-4578-9c5c-b2eb036a9c8c service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:03.905781 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdcbec-b4a5-4578-9c5c-b2eb036a9c8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:03.914681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fdcbec-b4a5-4578-9c5c-b2eb036a9c8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:03.916292 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fdcbec-b4a5-4578-9c5c-b2eb036a9c8c service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:03.929142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1224/2455] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:38:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:38:07.674937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840bec2-891d-4344-a56b-56f4c37e50a9 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:07.675426 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840bec2-891d-4344-a56b-56f4c37e50a9 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:07.677430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840bec2-891d-4344-a56b-56f4c37e50a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:07.693473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-0840bec2-891d-4344-a56b-56f4c37e50a9 None None] MFA Rules not processed for user `0235b8fe92554131ba30e1d18f0f2d86`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:07.694724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0840bec2-891d-4344-a56b-56f4c37e50a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:07.701546 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1232/2456] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:38:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:13.924930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=77,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:17.688412 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:17.971525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c45f45-1055-4a13-9eb2-1d5b0a953b8d None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:17.973892 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c45f45-1055-4a13-9eb2-1d5b0a953b8d None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:17.973892 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c45f45-1055-4a13-9eb2-1d5b0a953b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:17.989970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-21c45f45-1055-4a13-9eb2-1d5b0a953b8d None None] MFA Rules not processed for user `4e75a82cc7e94702ba0080fc65d6eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:17.991262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-21c45f45-1055-4a13-9eb2-1d5b0a953b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:18.051549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1225/2457] 10.222.0.78 () {58 vars in 967 bytes} [Wed Jul 20 15:38:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:18.226359 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840bec2-891d-4344-a56b-56f4c37e50a9 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:18.227787 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0840bec2-891d-4344-a56b-56f4c37e50a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:18.307917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840bec2-891d-4344-a56b-56f4c37e50a9 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:18.310843 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7e680-f590-4540-bad8-883d0d849602 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0d719c0eb04a3b8a228c6ce70fdf98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:18.311517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7e680-f590-4540-bad8-883d0d849602 service swift] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:18.311786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7e680-f590-4540-bad8-883d0d849602 service swift] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:18.312060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7e680-f590-4540-bad8-883d0d849602 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:18.327497 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa7e680-f590-4540-bad8-883d0d849602 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:18.330697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa7e680-f590-4540-bad8-883d0d849602 service swift] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:18.339991 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1233/2458] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Jul 20 15:38:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:38:19.327199 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c45f45-1055-4a13-9eb2-1d5b0a953b8d None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:19.341184 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c45f45-1055-4a13-9eb2-1d5b0a953b8d None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:19.350555 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131ed85-03b4-4a3f-9df6-98395af71613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:19.350555 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131ed85-03b4-4a3f-9df6-98395af71613 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:19.350555 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131ed85-03b4-4a3f-9df6-98395af71613 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:19.350555 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131ed85-03b4-4a3f-9df6-98395af71613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:19.351042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-2131ed85-03b4-4a3f-9df6-98395af71613 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:19.363335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131ed85-03b4-4a3f-9df6-98395af71613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:19.364915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131ed85-03b4-4a3f-9df6-98395af71613 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:19.376890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1226/2459] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:38:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:38:23.135825 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7e680-f590-4540-bad8-883d0d849602 service swift] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:23.138961 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-eaa7e680-f590-4540-bad8-883d0d849602 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:23.161908 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7e680-f590-4540-bad8-883d0d849602 service swift] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:23.167309 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465a8d5-37a5-4245-8786-e0cf4c74aa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:23.168530 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465a8d5-37a5-4245-8786-e0cf4c74aa0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:23.169077 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465a8d5-37a5-4245-8786-e0cf4c74aa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:23.169573 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465a8d5-37a5-4245-8786-e0cf4c74aa0a admin admin] PATH_INFO: `/v3/users/9ea2abd1398f4fba93b3b7a5beb52ba0` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:23.171028 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465a8d5-37a5-4245-8786-e0cf4c74aa0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ea2abd1398f4fba93b3b7a5beb52ba0)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:23.173383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465a8d5-37a5-4245-8786-e0cf4c74aa0a admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:23.459040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-2465a8d5-37a5-4245-8786-e0cf4c74aa0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ea2abd1398f4fba93b3b7a5beb52ba0'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:23.621994 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-2465a8d5-37a5-4245-8786-e0cf4c74aa0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ea2abd1398f4fba93b3b7a5beb52ba0'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:23.630222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-2465a8d5-37a5-4245-8786-e0cf4c74aa0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ea2abd1398f4fba93b3b7a5beb52ba0'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:23.630982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-2465a8d5-37a5-4245-8786-e0cf4c74aa0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ea2abd1398f4fba93b3b7a5beb52ba0'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:23.647450 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-2465a8d5-37a5-4245-8786-e0cf4c74aa0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ea2abd1398f4fba93b3b7a5beb52ba0'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:23.657519 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1234/2460] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:38:23 2022] DELETE /identity/v3/users/9ea2abd1398f4fba93b3b7a5beb52ba0 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:24.186280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131ed85-03b4-4a3f-9df6-98395af71613 service nova] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:24.197022 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131ed85-03b4-4a3f-9df6-98395af71613 service nova] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:24.286734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:24.287615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:24.287896 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:24.288169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979 admin admin] PATH_INFO: `/v3/projects/3e3b499bb69b4b449a2e2515d94094fb` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:24.289542 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e3b499bb69b4b449a2e2515d94094fb)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:24.291652 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:24.485724 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e3b499bb69b4b449a2e2515d94094fb', 'request_id': 'req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:24.650626 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e3b499bb69b4b449a2e2515d94094fb', 'request_id': 'req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:24.659685 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e3b499bb69b4b449a2e2515d94094fb', 'request_id': 'req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:24.695938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979 admin admin] The token cache is being invalidate because project 3e3b499bb69b4b449a2e2515d94094fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:38:24.696407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:24.697250 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:24.723243 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1227/2461] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:38:24 2022] DELETE /identity/v3/projects/3e3b499bb69b4b449a2e2515d94094fb => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:25.009360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465a8d5-37a5-4245-8786-e0cf4c74aa0a admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:25.013615 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2465a8d5-37a5-4245-8786-e0cf4c74aa0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:25.130640 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:25.135980 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465a8d5-37a5-4245-8786-e0cf4c74aa0a admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:25.213110 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a024c0-1db2-42f9-b893-6440968768e4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2192547babde4449a46f9cb37ed64524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:25.214483 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a024c0-1db2-42f9-b893-6440968768e4 service placement] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:25.214673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a024c0-1db2-42f9-b893-6440968768e4 service placement] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:25.214867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a024c0-1db2-42f9-b893-6440968768e4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:25.217020 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-93a024c0-1db2-42f9-b893-6440968768e4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:25.231088 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a024c0-1db2-42f9-b893-6440968768e4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:25.235182 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a024c0-1db2-42f9-b893-6440968768e4 service placement] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:25.250078 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1235/2462] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:38:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:38:29.849477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:29.850961 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:29.879196 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:29.920964 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc5f41-ce80-4a49-9e61-bb17de9f7979 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:29.926136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b82a92-cefc-4aa1-9faa-8619f3eb53dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:29.927080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b82a92-cefc-4aa1-9faa-8619f3eb53dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:29.927370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b82a92-cefc-4aa1-9faa-8619f3eb53dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:29.927647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b82a92-cefc-4aa1-9faa-8619f3eb53dd admin admin] PATH_INFO: `/v3/users/dce92cce74c84abcbec76358c860535f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:29.928888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b82a92-cefc-4aa1-9faa-8619f3eb53dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=dce92cce74c84abcbec76358c860535f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:29.930856 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b82a92-cefc-4aa1-9faa-8619f3eb53dd admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:30.175039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-b9b82a92-cefc-4aa1-9faa-8619f3eb53dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dce92cce74c84abcbec76358c860535f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:30.284256 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-b9b82a92-cefc-4aa1-9faa-8619f3eb53dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dce92cce74c84abcbec76358c860535f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:30.284875 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-b9b82a92-cefc-4aa1-9faa-8619f3eb53dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dce92cce74c84abcbec76358c860535f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:30.298056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-b9b82a92-cefc-4aa1-9faa-8619f3eb53dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dce92cce74c84abcbec76358c860535f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:30.305227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-b9b82a92-cefc-4aa1-9faa-8619f3eb53dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dce92cce74c84abcbec76358c860535f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:30.313203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1228/2463] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:38:29 2022] DELETE /identity/v3/users/dce92cce74c84abcbec76358c860535f => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:30.591847 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a024c0-1db2-42f9-b893-6440968768e4 service placement] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:30.603283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a024c0-1db2-42f9-b893-6440968768e4 service placement] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:30.668131 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd5c35-af14-4b54-afe8-8ef31fc08845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:30.669044 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd5c35-af14-4b54-afe8-8ef31fc08845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:30.669332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd5c35-af14-4b54-afe8-8ef31fc08845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:30.669615 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd5c35-af14-4b54-afe8-8ef31fc08845 admin admin] PATH_INFO: `/v3/projects/8793a880033e400d89ac8de5a30003b7` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:30.670849 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd5c35-af14-4b54-afe8-8ef31fc08845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8793a880033e400d89ac8de5a30003b7)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:30.672785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd5c35-af14-4b54-afe8-8ef31fc08845 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:30.809080 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-1fcd5c35-af14-4b54-afe8-8ef31fc08845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8793a880033e400d89ac8de5a30003b7', 'request_id': 'req-1fcd5c35-af14-4b54-afe8-8ef31fc08845'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:30.915669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-1fcd5c35-af14-4b54-afe8-8ef31fc08845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8793a880033e400d89ac8de5a30003b7', 'request_id': 'req-1fcd5c35-af14-4b54-afe8-8ef31fc08845'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:30.922664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-1fcd5c35-af14-4b54-afe8-8ef31fc08845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8793a880033e400d89ac8de5a30003b7', 'request_id': 'req-1fcd5c35-af14-4b54-afe8-8ef31fc08845'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:30.933842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-1fcd5c35-af14-4b54-afe8-8ef31fc08845 admin admin] The token cache is being invalidate because project 8793a880033e400d89ac8de5a30003b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:38:30.934124 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-1fcd5c35-af14-4b54-afe8-8ef31fc08845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:30.934802 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-1fcd5c35-af14-4b54-afe8-8ef31fc08845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:30.936387 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1236/2464] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:38:30 2022] DELETE /identity/v3/projects/8793a880033e400d89ac8de5a30003b7 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:30.940678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b82a92-cefc-4aa1-9faa-8619f3eb53dd admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:30.942110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b9b82a92-cefc-4aa1-9faa-8619f3eb53dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:30.957047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b82a92-cefc-4aa1-9faa-8619f3eb53dd admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:31.015814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558d5fa-4906-470c-955c-7f282cf5a198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:31.021393 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558d5fa-4906-470c-955c-7f282cf5a198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:31.021393 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558d5fa-4906-470c-955c-7f282cf5a198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:31.021393 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558d5fa-4906-470c-955c-7f282cf5a198 admin admin] PATH_INFO: `/v3/users/d8a92f26477b40dcaf77913e55c5fa6b` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:31.021393 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558d5fa-4906-470c-955c-7f282cf5a198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8a92f26477b40dcaf77913e55c5fa6b)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:31.021393 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558d5fa-4906-470c-955c-7f282cf5a198 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:31.292894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:31.295681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-0558d5fa-4906-470c-955c-7f282cf5a198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8a92f26477b40dcaf77913e55c5fa6b'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:31.438128 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-0558d5fa-4906-470c-955c-7f282cf5a198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8a92f26477b40dcaf77913e55c5fa6b'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:31.438812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-0558d5fa-4906-470c-955c-7f282cf5a198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8a92f26477b40dcaf77913e55c5fa6b'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:31.451907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-0558d5fa-4906-470c-955c-7f282cf5a198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8a92f26477b40dcaf77913e55c5fa6b'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:31.459637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-0558d5fa-4906-470c-955c-7f282cf5a198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8a92f26477b40dcaf77913e55c5fa6b'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:31.465784 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1229/2465] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:38:30 2022] DELETE /identity/v3/users/d8a92f26477b40dcaf77913e55c5fa6b => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:31.757536 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd5c35-af14-4b54-afe8-8ef31fc08845 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:31.759688 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1fcd5c35-af14-4b54-afe8-8ef31fc08845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:31.772969 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd5c35-af14-4b54-afe8-8ef31fc08845 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:31.795140 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:31.833487 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8984b82-94c9-4b8e-ba73-39492af6c29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:31.834291 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8984b82-94c9-4b8e-ba73-39492af6c29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:31.834587 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8984b82-94c9-4b8e-ba73-39492af6c29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:31.834863 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8984b82-94c9-4b8e-ba73-39492af6c29f admin admin] PATH_INFO: `/v3/projects/d33d090d45b04ef99ceae6dd4f806320` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:31.836157 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8984b82-94c9-4b8e-ba73-39492af6c29f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d33d090d45b04ef99ceae6dd4f806320)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:31.838240 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8984b82-94c9-4b8e-ba73-39492af6c29f admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:32.069974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c8984b82-94c9-4b8e-ba73-39492af6c29f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd33d090d45b04ef99ceae6dd4f806320', 'request_id': 'req-c8984b82-94c9-4b8e-ba73-39492af6c29f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:32.169691 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c8984b82-94c9-4b8e-ba73-39492af6c29f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd33d090d45b04ef99ceae6dd4f806320', 'request_id': 'req-c8984b82-94c9-4b8e-ba73-39492af6c29f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:32.176858 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c8984b82-94c9-4b8e-ba73-39492af6c29f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd33d090d45b04ef99ceae6dd4f806320', 'request_id': 'req-c8984b82-94c9-4b8e-ba73-39492af6c29f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:32.187017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c8984b82-94c9-4b8e-ba73-39492af6c29f admin admin] The token cache is being invalidate because project d33d090d45b04ef99ceae6dd4f806320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:38:32.187309 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c8984b82-94c9-4b8e-ba73-39492af6c29f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:32.187943 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c8984b82-94c9-4b8e-ba73-39492af6c29f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:38:32.189538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1237/2466] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:38:31 2022] DELETE /identity/v3/projects/d33d090d45b04ef99ceae6dd4f806320 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:38:41.544240 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:DELETE=4,keystone:SELECT=13,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:42.361651 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:47.989411 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac151ac3-fe7c-432a-9a39-0b1fcea395bf None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:47.989873 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac151ac3-fe7c-432a-9a39-0b1fcea395bf None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:47.989981 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac151ac3-fe7c-432a-9a39-0b1fcea395bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:48.008572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-ac151ac3-fe7c-432a-9a39-0b1fcea395bf None None] MFA Rules not processed for user `4e75a82cc7e94702ba0080fc65d6eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:38:48.009777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ac151ac3-fe7c-432a-9a39-0b1fcea395bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:48.064883 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1230/2467] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Jul 20 15:38:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:38:48.077029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8984b82-94c9-4b8e-ba73-39492af6c29f admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:38:48.078690 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c8984b82-94c9-4b8e-ba73-39492af6c29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:38:48.150665 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8984b82-94c9-4b8e-ba73-39492af6c29f admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:38:48.156210 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e8624-eab2-46fa-b77b-a98b2d5825f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0d719c0eb04a3b8a228c6ce70fdf98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:38:48.156850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e8624-eab2-46fa-b77b-a98b2d5825f8 service swift] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:38:48.157135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e8624-eab2-46fa-b77b-a98b2d5825f8 service swift] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:38:48.157411 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e8624-eab2-46fa-b77b-a98b2d5825f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:38:48.165526 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e8624-eab2-46fa-b77b-a98b2d5825f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:38:48.173162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e8624-eab2-46fa-b77b-a98b2d5825f8 service swift] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:38:48.175080 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1238/2468] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Jul 20 15:38:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:38:58.058240 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=25 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:38:58.171932 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=32 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:05.574149 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac151ac3-fe7c-432a-9a39-0b1fcea395bf None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:05.575586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ac151ac3-fe7c-432a-9a39-0b1fcea395bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:05.633878 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac151ac3-fe7c-432a-9a39-0b1fcea395bf None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:05.637718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d99a8b-a676-4e02-9c7b-27d3ca9e61b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:05.638406 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d99a8b-a676-4e02-9c7b-27d3ca9e61b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:05.638684 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d99a8b-a676-4e02-9c7b-27d3ca9e61b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:05.638959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d99a8b-a676-4e02-9c7b-27d3ca9e61b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:05.645361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d5d99a8b-a676-4e02-9c7b-27d3ca9e61b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:05.706152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d99a8b-a676-4e02-9c7b-27d3ca9e61b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:05.707637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d99a8b-a676-4e02-9c7b-27d3ca9e61b3 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:05.718479 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1231/2469] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:08.098457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b62c1-607c-44d5-bda9-423338db2275 None None] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:08.098457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b62c1-607c-44d5-bda9-423338db2275 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:08.098457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b62c1-607c-44d5-bda9-423338db2275 None None] PATH_INFO: `/` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:08.099802 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1239/2470] 10.222.0.78 () {54 vars in 921 bytes} [Wed Jul 20 15:39:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:39:08.105523 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668fada-5627-4d8d-88ec-331aee398dd2 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:08.106037 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668fada-5627-4d8d-88ec-331aee398dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:08.106140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668fada-5627-4d8d-88ec-331aee398dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:08.127302 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-9668fada-5627-4d8d-88ec-331aee398dd2 None None] MFA Rules not processed for user `b57530f18d9c4d328f8014db736d8de9`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:08.128911 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-9668fada-5627-4d8d-88ec-331aee398dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:08.143298 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1232/2471] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Jul 20 15:39:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:08.149309 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b62c1-607c-44d5-bda9-423338db2275 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:08.155857 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8f6b62c1-607c-44d5-bda9-423338db2275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:08.175922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b62c1-607c-44d5-bda9-423338db2275 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:08.180350 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec35c0-d701-4562-9bd8-3ffcd81eaef8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:08.181254 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec35c0-d701-4562-9bd8-3ffcd81eaef8 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:08.181709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec35c0-d701-4562-9bd8-3ffcd81eaef8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:08.182701 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec35c0-d701-4562-9bd8-3ffcd81eaef8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:08.191694 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec35c0-d701-4562-9bd8-3ffcd81eaef8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:08.194185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec35c0-d701-4562-9bd8-3ffcd81eaef8 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:08.204350 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1240/2472] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:08.300954 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606a4e6-83d0-4fb0-bdac-7f49c1e963ff None None] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:08.301511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606a4e6-83d0-4fb0-bdac-7f49c1e963ff None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:08.301941 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606a4e6-83d0-4fb0-bdac-7f49c1e963ff None None] PATH_INFO: `/` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:08.304556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1233/2473] 10.222.0.78 () {54 vars in 921 bytes} [Wed Jul 20 15:39:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 15:39:08.309434 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c524b0b-cae8-4e97-a046-740eaf8ed8ad None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:08.310347 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c524b0b-cae8-4e97-a046-740eaf8ed8ad None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:08.310854 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c524b0b-cae8-4e97-a046-740eaf8ed8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:08.338297 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-8c524b0b-cae8-4e97-a046-740eaf8ed8ad None None] MFA Rules not processed for user `a4a24d5a375945779fa626fae25ac237`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:08.340770 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8c524b0b-cae8-4e97-a046-740eaf8ed8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:08.431694 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1241/2474] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Jul 20 15:39:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:39:08.442652 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606a4e6-83d0-4fb0-bdac-7f49c1e963ff None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:08.454052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606a4e6-83d0-4fb0-bdac-7f49c1e963ff None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:08.458644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d0da0-03b4-49aa-9f58-142463ef619b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:08.464233 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d0da0-03b4-49aa-9f58-142463ef619b service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:08.464233 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d0da0-03b4-49aa-9f58-142463ef619b service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:08.464233 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d0da0-03b4-49aa-9f58-142463ef619b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:08.473121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d0da0-03b4-49aa-9f58-142463ef619b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:08.475318 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d0da0-03b4-49aa-9f58-142463ef619b service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:08.486890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1234/2475] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:39:10.330628 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c524b0b-cae8-4e97-a046-740eaf8ed8ad None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:10.332400 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8c524b0b-cae8-4e97-a046-740eaf8ed8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:10.350633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c524b0b-cae8-4e97-a046-740eaf8ed8ad None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:10.356088 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac07e8-e8a5-4944-8412-fde8fcc1d7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:10.357229 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac07e8-e8a5-4944-8412-fde8fcc1d7be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:10.357507 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac07e8-e8a5-4944-8412-fde8fcc1d7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:10.357786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac07e8-e8a5-4944-8412-fde8fcc1d7be admin admin] PATH_INFO: `/v3/users/8e75c180bae74be0b2ccad079f92d32b` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:10.359040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac07e8-e8a5-4944-8412-fde8fcc1d7be admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e75c180bae74be0b2ccad079f92d32b)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:10.362070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac07e8-e8a5-4944-8412-fde8fcc1d7be admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:10.585279 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-d8ac07e8-e8a5-4944-8412-fde8fcc1d7be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e75c180bae74be0b2ccad079f92d32b'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:10.674420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-d8ac07e8-e8a5-4944-8412-fde8fcc1d7be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e75c180bae74be0b2ccad079f92d32b'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:10.685861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-d8ac07e8-e8a5-4944-8412-fde8fcc1d7be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e75c180bae74be0b2ccad079f92d32b'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:10.686739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-d8ac07e8-e8a5-4944-8412-fde8fcc1d7be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e75c180bae74be0b2ccad079f92d32b'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:10.700770 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-d8ac07e8-e8a5-4944-8412-fde8fcc1d7be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e75c180bae74be0b2ccad079f92d32b'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:10.707206 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1242/2476] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:39:10 2022] DELETE /identity/v3/users/8e75c180bae74be0b2ccad079f92d32b => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:10.989845 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d0da0-03b4-49aa-9f58-142463ef619b service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:11.010965 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d0da0-03b4-49aa-9f58-142463ef619b service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:11.069858 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e7f9e-c347-4f24-980e-7c18255ed381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:11.070664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e7f9e-c347-4f24-980e-7c18255ed381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:11.070955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e7f9e-c347-4f24-980e-7c18255ed381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:11.071231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e7f9e-c347-4f24-980e-7c18255ed381 admin admin] PATH_INFO: `/v3/projects/aa49d7002cf441348cf3166e64b1bc87` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:11.072411 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e7f9e-c347-4f24-980e-7c18255ed381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa49d7002cf441348cf3166e64b1bc87)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:11.074927 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e7f9e-c347-4f24-980e-7c18255ed381 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:11.192943 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:11.209396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-505e7f9e-c347-4f24-980e-7c18255ed381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa49d7002cf441348cf3166e64b1bc87', 'request_id': 'req-505e7f9e-c347-4f24-980e-7c18255ed381'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:11.283593 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-505e7f9e-c347-4f24-980e-7c18255ed381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa49d7002cf441348cf3166e64b1bc87', 'request_id': 'req-505e7f9e-c347-4f24-980e-7c18255ed381'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:11.291852 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-505e7f9e-c347-4f24-980e-7c18255ed381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa49d7002cf441348cf3166e64b1bc87', 'request_id': 'req-505e7f9e-c347-4f24-980e-7c18255ed381'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:11.306853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-505e7f9e-c347-4f24-980e-7c18255ed381 admin admin] The token cache is being invalidate because project aa49d7002cf441348cf3166e64b1bc87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:39:11.307678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-505e7f9e-c347-4f24-980e-7c18255ed381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:11.308348 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-505e7f9e-c347-4f24-980e-7c18255ed381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:11.310222 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1235/2477] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:39:10 2022] DELETE /identity/v3/projects/aa49d7002cf441348cf3166e64b1bc87 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:11.316759 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a0849-658c-4e9c-860a-c7c6be300be5 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:11.317035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a0849-658c-4e9c-860a-c7c6be300be5 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:11.317326 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a0849-658c-4e9c-860a-c7c6be300be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:11.337154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-038a0849-658c-4e9c-860a-c7c6be300be5 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:11.338306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-038a0849-658c-4e9c-860a-c7c6be300be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:11.387293 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:11.406974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1243/2478] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:11.415087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cada50-6064-441b-994b-32202a365c54 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:11.415426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cada50-6064-441b-994b-32202a365c54 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:11.415710 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cada50-6064-441b-994b-32202a365c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:11.440156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-e2cada50-6064-441b-994b-32202a365c54 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:11.441638 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e2cada50-6064-441b-994b-32202a365c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:11.449865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1236/2479] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:11.454173 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a0849-658c-4e9c-860a-c7c6be300be5 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:11.464935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a0849-658c-4e9c-860a-c7c6be300be5 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:11.470245 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d93d60-c8fd-4ad3-9de5-4c6428ccea61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:11.471026 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d93d60-c8fd-4ad3-9de5-4c6428ccea61 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:11.471381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d93d60-c8fd-4ad3-9de5-4c6428ccea61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:11.471833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d93d60-c8fd-4ad3-9de5-4c6428ccea61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:11.472699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d93d60-c8fd-4ad3-9de5-4c6428ccea61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:39:11.473012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d93d60-c8fd-4ad3-9de5-4c6428ccea61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:11.474765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d93d60-c8fd-4ad3-9de5-4c6428ccea61 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:11.487851 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1244/2480] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:39:11.492571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cada50-6064-441b-994b-32202a365c54 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:11.505740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cada50-6064-441b-994b-32202a365c54 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:11.510808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60833be-c96f-4cd1-938e-7a1a320877f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:11.511541 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60833be-c96f-4cd1-938e-7a1a320877f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:11.511894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60833be-c96f-4cd1-938e-7a1a320877f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:11.512256 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60833be-c96f-4cd1-938e-7a1a320877f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:11.513023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60833be-c96f-4cd1-938e-7a1a320877f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:11.515017 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60833be-c96f-4cd1-938e-7a1a320877f0 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:11.665545 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1237/2481] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:39:11 2022] POST /identity/v3/projects => generated 403 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:39:11.670990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d93d60-c8fd-4ad3-9de5-4c6428ccea61 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:11.681666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d93d60-c8fd-4ad3-9de5-4c6428ccea61 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:11.742117 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe56944-7f3c-4c39-a263-951cd1344d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:11.742837 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe56944-7f3c-4c39-a263-951cd1344d11 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:11.743119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe56944-7f3c-4c39-a263-951cd1344d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:11.743392 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe56944-7f3c-4c39-a263-951cd1344d11 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:11.744076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe56944-7f3c-4c39-a263-951cd1344d11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:11.746110 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe56944-7f3c-4c39-a263-951cd1344d11 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:11.864194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1245/2482] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:39:11 2022] POST /identity/v3/users => generated 302 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:11.868244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60833be-c96f-4cd1-938e-7a1a320877f0 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:11.879626 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60833be-c96f-4cd1-938e-7a1a320877f0 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:11.884751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e0ebf-a13a-4640-9b10-cb06164f933d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:11.885466 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e0ebf-a13a-4640-9b10-cb06164f933d admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:11.885774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e0ebf-a13a-4640-9b10-cb06164f933d admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:11.886025 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e0ebf-a13a-4640-9b10-cb06164f933d admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:11.886807 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e0ebf-a13a-4640-9b10-cb06164f933d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:11.888487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e0ebf-a13a-4640-9b10-cb06164f933d admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:11.911394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1238/2483] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:11.913711 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe56944-7f3c-4c39-a263-951cd1344d11 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:11.928482 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe56944-7f3c-4c39-a263-951cd1344d11 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:11.934459 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e5c67-ac79-4b61-9bc3-c508763ac710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:11.934459 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e5c67-ac79-4b61-9bc3-c508763ac710 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:11.934459 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e5c67-ac79-4b61-9bc3-c508763ac710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:11.934930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e5c67-ac79-4b61-9bc3-c508763ac710 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:11.935463 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e5c67-ac79-4b61-9bc3-c508763ac710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:11.937715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e5c67-ac79-4b61-9bc3-c508763ac710 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:11.948797 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1246/2484] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:11.953043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e0ebf-a13a-4640-9b10-cb06164f933d admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:11.965055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e0ebf-a13a-4640-9b10-cb06164f933d admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:11.970544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71c622-8c30-49d3-bd57-ff37202d69c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:11.971642 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71c622-8c30-49d3-bd57-ff37202d69c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:11.971939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71c622-8c30-49d3-bd57-ff37202d69c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:11.972301 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71c622-8c30-49d3-bd57-ff37202d69c0 admin admin] PATH_INFO: `/v3/projects/48211c3f11ca415bb8decbc5bf4f4ef4/users/d8c88e23f75348f4b13b3a36eb37962c/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:11.992283 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71c622-8c30-49d3-bd57-ff37202d69c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48211c3f11ca415bb8decbc5bf4f4ef4, user_id=d8c88e23f75348f4b13b3a36eb37962c, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:11.996015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71c622-8c30-49d3-bd57-ff37202d69c0 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:12.105554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1239/2485] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:11 2022] PUT /identity/v3/projects/48211c3f11ca415bb8decbc5bf4f4ef4/users/d8c88e23f75348f4b13b3a36eb37962c/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:39:12.116259 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b2d92-a8f5-4b2f-8ac0-a0efe1487e02 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:12.116500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b2d92-a8f5-4b2f-8ac0-a0efe1487e02 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:12.116784 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b2d92-a8f5-4b2f-8ac0-a0efe1487e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:12.147562 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-ec2b2d92-a8f5-4b2f-8ac0-a0efe1487e02 None None] MFA Rules not processed for user `d8c88e23f75348f4b13b3a36eb37962c`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:12.148829 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ec2b2d92-a8f5-4b2f-8ac0-a0efe1487e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:12.159456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01941a-f965-4a25-9b78-fad51900eea1 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:12.159850 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01941a-f965-4a25-9b78-fad51900eea1 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:12.160337 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01941a-f965-4a25-9b78-fad51900eea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:12.179422 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-ef01941a-f965-4a25-9b78-fad51900eea1 None None] MFA Rules not processed for user `4e75a82cc7e94702ba0080fc65d6eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:12.180726 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ef01941a-f965-4a25-9b78-fad51900eea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:12.236484 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1247/2486] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:12.241266 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b2d92-a8f5-4b2f-8ac0-a0efe1487e02 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:12.243745 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1240/2487] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Jul 20 15:39:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:39:12.256070 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01941a-f965-4a25-9b78-fad51900eea1 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:12.257536 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ef01941a-f965-4a25-9b78-fad51900eea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:12.293308 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:12.308724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b2d92-a8f5-4b2f-8ac0-a0efe1487e02 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:12.313826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9e411-cafd-45a6-9087-e49447bf851a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:12.314528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9e411-cafd-45a6-9087-e49447bf851a admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:12.314814 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9e411-cafd-45a6-9087-e49447bf851a admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:12.315093 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9e411-cafd-45a6-9087-e49447bf851a admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:12.315781 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9e411-cafd-45a6-9087-e49447bf851a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:12.317802 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9e411-cafd-45a6-9087-e49447bf851a admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:12.328274 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:12.334485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01941a-f965-4a25-9b78-fad51900eea1 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:12.342616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1d029-d00d-4475-994c-bbc61accb4e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0d719c0eb04a3b8a228c6ce70fdf98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:12.343495 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1d029-d00d-4475-994c-bbc61accb4e2 service swift] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:12.343827 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1d029-d00d-4475-994c-bbc61accb4e2 service swift] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:12.344174 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1d029-d00d-4475-994c-bbc61accb4e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:12.354746 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1d029-d00d-4475-994c-bbc61accb4e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:12.355918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1d029-d00d-4475-994c-bbc61accb4e2 service swift] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:12.364947 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1241/2488] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Jul 20 15:39:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 15:39:12.566111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1248/2489] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:39:12 2022] POST /identity/v3/projects => generated 405 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:12.570729 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1d029-d00d-4475-994c-bbc61accb4e2 service swift] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:12.583514 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1d029-d00d-4475-994c-bbc61accb4e2 service swift] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:12.714198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47954fc7-82c2-44c6-b946-bad9e3022884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:12.716449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47954fc7-82c2-44c6-b946-bad9e3022884 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:12.718085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47954fc7-82c2-44c6-b946-bad9e3022884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:12.720321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47954fc7-82c2-44c6-b946-bad9e3022884 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:12.722649 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47954fc7-82c2-44c6-b946-bad9e3022884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:12.728064 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47954fc7-82c2-44c6-b946-bad9e3022884 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:12.849177 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1242/2490] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:39:12 2022] POST /identity/v3/users => generated 309 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:12.851184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9e411-cafd-45a6-9087-e49447bf851a admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:12.862972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9e411-cafd-45a6-9087-e49447bf851a admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:12.868232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e7395-d555-4e84-9cd8-64d2b2c6762e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:12.869009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e7395-d555-4e84-9cd8-64d2b2c6762e admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:12.869312 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e7395-d555-4e84-9cd8-64d2b2c6762e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:12.870227 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e7395-d555-4e84-9cd8-64d2b2c6762e admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:12.871016 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e7395-d555-4e84-9cd8-64d2b2c6762e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:12.872754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e7395-d555-4e84-9cd8-64d2b2c6762e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:12.890426 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1249/2491] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:12.890625 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47954fc7-82c2-44c6-b946-bad9e3022884 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:12.906109 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47954fc7-82c2-44c6-b946-bad9e3022884 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:12.917301 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f910b3-632e-49cc-ae29-7bc76ddcacbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:12.918634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f910b3-632e-49cc-ae29-7bc76ddcacbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:12.919339 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f910b3-632e-49cc-ae29-7bc76ddcacbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:12.920529 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f910b3-632e-49cc-ae29-7bc76ddcacbf admin admin] PATH_INFO: `/v3/projects/72187061e9ba4275b81c7ee42529a1ad/users/fdda71044faf489ebe9ba5c4862b5768/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:12.942713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f910b3-632e-49cc-ae29-7bc76ddcacbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=72187061e9ba4275b81c7ee42529a1ad, user_id=fdda71044faf489ebe9ba5c4862b5768, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:12.945544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f910b3-632e-49cc-ae29-7bc76ddcacbf admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:13.059143 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1243/2492] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:12 2022] PUT /identity/v3/projects/72187061e9ba4275b81c7ee42529a1ad/users/fdda71044faf489ebe9ba5c4862b5768/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:13.065052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e7395-d555-4e84-9cd8-64d2b2c6762e admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:13.077941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e7395-d555-4e84-9cd8-64d2b2c6762e admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:13.234778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3a202-4abe-4b23-aa6c-d7761cbc2a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:13.235474 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3a202-4abe-4b23-aa6c-d7761cbc2a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:13.235760 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3a202-4abe-4b23-aa6c-d7761cbc2a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:13.236052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3a202-4abe-4b23-aa6c-d7761cbc2a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:13.236938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3a202-4abe-4b23-aa6c-d7761cbc2a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:13.238903 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3a202-4abe-4b23-aa6c-d7761cbc2a17 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:13.252795 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1250/2493] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:13.257150 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f910b3-632e-49cc-ae29-7bc76ddcacbf admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:13.270740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f910b3-632e-49cc-ae29-7bc76ddcacbf admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:13.277168 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153db2ea-361b-4dd2-a3fc-19d9a74c01e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:13.278078 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153db2ea-361b-4dd2-a3fc-19d9a74c01e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:13.278569 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153db2ea-361b-4dd2-a3fc-19d9a74c01e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:13.279024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153db2ea-361b-4dd2-a3fc-19d9a74c01e2 admin admin] PATH_INFO: `/v3/projects/72187061e9ba4275b81c7ee42529a1ad/users/fdda71044faf489ebe9ba5c4862b5768/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:13.281457 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153db2ea-361b-4dd2-a3fc-19d9a74c01e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72187061e9ba4275b81c7ee42529a1ad, user_id=fdda71044faf489ebe9ba5c4862b5768, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:13.284344 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153db2ea-361b-4dd2-a3fc-19d9a74c01e2 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:13.312532 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1244/2494] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:13 2022] PUT /identity/v3/projects/72187061e9ba4275b81c7ee42529a1ad/users/fdda71044faf489ebe9ba5c4862b5768/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:13.317614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38dec0-154c-49b3-a739-0eaba0bde99f None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:13.318132 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38dec0-154c-49b3-a739-0eaba0bde99f None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:13.318997 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38dec0-154c-49b3-a739-0eaba0bde99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:13.378768 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-cb38dec0-154c-49b3-a739-0eaba0bde99f None None] MFA Rules not processed for user `fdda71044faf489ebe9ba5c4862b5768`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:13.380026 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-cb38dec0-154c-49b3-a739-0eaba0bde99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:13.464965 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1251/2495] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:13 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:13.477152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153db2ea-361b-4dd2-a3fc-19d9a74c01e2 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:13.479549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-153db2ea-361b-4dd2-a3fc-19d9a74c01e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:13.582606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153db2ea-361b-4dd2-a3fc-19d9a74c01e2 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:13.590180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38497c0c-b826-44f9-8167-4186a803662e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:13.591706 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38497c0c-b826-44f9-8167-4186a803662e service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:13.592291 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38497c0c-b826-44f9-8167-4186a803662e service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:13.592730 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38497c0c-b826-44f9-8167-4186a803662e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:13.617919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38497c0c-b826-44f9-8167-4186a803662e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:13.619404 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38497c0c-b826-44f9-8167-4186a803662e service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:13.631721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1245/2496] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:13 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:13.789657 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38dec0-154c-49b3-a739-0eaba0bde99f None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:13.851317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:13.874003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38dec0-154c-49b3-a739-0eaba0bde99f None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:13.879199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3380135-d6d0-4ede-8b9e-246714c2d0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:13.880263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3380135-d6d0-4ede-8b9e-246714c2d0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:13.880538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3380135-d6d0-4ede-8b9e-246714c2d0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:13.880899 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3380135-d6d0-4ede-8b9e-246714c2d0b4 admin admin] PATH_INFO: `/v3/users/d8c88e23f75348f4b13b3a36eb37962c` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:13.882173 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3380135-d6d0-4ede-8b9e-246714c2d0b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c88e23f75348f4b13b3a36eb37962c)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:13.884257 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3380135-d6d0-4ede-8b9e-246714c2d0b4 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:14.056629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c3380135-d6d0-4ede-8b9e-246714c2d0b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c88e23f75348f4b13b3a36eb37962c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:14.141171 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c3380135-d6d0-4ede-8b9e-246714c2d0b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c88e23f75348f4b13b3a36eb37962c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:14.151700 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c3380135-d6d0-4ede-8b9e-246714c2d0b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c88e23f75348f4b13b3a36eb37962c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:14.152793 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c3380135-d6d0-4ede-8b9e-246714c2d0b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c88e23f75348f4b13b3a36eb37962c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:14.171309 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c3380135-d6d0-4ede-8b9e-246714c2d0b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c88e23f75348f4b13b3a36eb37962c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:14.178216 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1252/2497] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:39:13 2022] DELETE /identity/v3/users/d8c88e23f75348f4b13b3a36eb37962c => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:14.182606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb27b47-3ae5-482b-9ae9-a300ed05ee5a None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:14.183123 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb27b47-3ae5-482b-9ae9-a300ed05ee5a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:14.183220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb27b47-3ae5-482b-9ae9-a300ed05ee5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:14.203268 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-0bb27b47-3ae5-482b-9ae9-a300ed05ee5a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:14.204584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0bb27b47-3ae5-482b-9ae9-a300ed05ee5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:14.252835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:14.270863 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1246/2498] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:14.283990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3380135-d6d0-4ede-8b9e-246714c2d0b4 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:14.286478 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c3380135-d6d0-4ede-8b9e-246714c2d0b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:14.361243 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3380135-d6d0-4ede-8b9e-246714c2d0b4 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:14.372571 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d1555-364e-4f93-963f-8acd5c38cadd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:14.373468 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d1555-364e-4f93-963f-8acd5c38cadd service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:14.375504 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d1555-364e-4f93-963f-8acd5c38cadd service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:14.375504 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d1555-364e-4f93-963f-8acd5c38cadd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:14.389510 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d1555-364e-4f93-963f-8acd5c38cadd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:14.391922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d1555-364e-4f93-963f-8acd5c38cadd service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:14.422192 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1253/2499] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:14.996976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb27b47-3ae5-482b-9ae9-a300ed05ee5a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:14.998716 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0bb27b47-3ae5-482b-9ae9-a300ed05ee5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:15.018633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb27b47-3ae5-482b-9ae9-a300ed05ee5a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:15.026695 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01226c9a-3286-4496-bf65-61cd2e9ddb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:15.028606 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01226c9a-3286-4496-bf65-61cd2e9ddb57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:15.029890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01226c9a-3286-4496-bf65-61cd2e9ddb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:15.030377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01226c9a-3286-4496-bf65-61cd2e9ddb57 admin admin] PATH_INFO: `/v3/projects/48211c3f11ca415bb8decbc5bf4f4ef4` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:15.032091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01226c9a-3286-4496-bf65-61cd2e9ddb57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48211c3f11ca415bb8decbc5bf4f4ef4)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:15.034631 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01226c9a-3286-4496-bf65-61cd2e9ddb57 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:15.259701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-01226c9a-3286-4496-bf65-61cd2e9ddb57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48211c3f11ca415bb8decbc5bf4f4ef4', 'request_id': 'req-01226c9a-3286-4496-bf65-61cd2e9ddb57'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:15.468362 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-01226c9a-3286-4496-bf65-61cd2e9ddb57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48211c3f11ca415bb8decbc5bf4f4ef4', 'request_id': 'req-01226c9a-3286-4496-bf65-61cd2e9ddb57'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:15.475260 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-01226c9a-3286-4496-bf65-61cd2e9ddb57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48211c3f11ca415bb8decbc5bf4f4ef4', 'request_id': 'req-01226c9a-3286-4496-bf65-61cd2e9ddb57'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:15.487146 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-01226c9a-3286-4496-bf65-61cd2e9ddb57 admin admin] The token cache is being invalidate because project 48211c3f11ca415bb8decbc5bf4f4ef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:39:15.487432 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-01226c9a-3286-4496-bf65-61cd2e9ddb57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:15.488082 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-01226c9a-3286-4496-bf65-61cd2e9ddb57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:15.489675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1247/2500] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:39:14 2022] DELETE /identity/v3/projects/48211c3f11ca415bb8decbc5bf4f4ef4 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:15.493622 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d1555-364e-4f93-963f-8acd5c38cadd service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:15.503742 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d1555-364e-4f93-963f-8acd5c38cadd service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:15.578543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0420a-002a-47d5-be64-bfaa4cd11fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:15.579773 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0420a-002a-47d5-be64-bfaa4cd11fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:15.579773 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0420a-002a-47d5-be64-bfaa4cd11fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:15.580099 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0420a-002a-47d5-be64-bfaa4cd11fca admin admin] PATH_INFO: `/v3/users/fdda71044faf489ebe9ba5c4862b5768` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:15.581402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d0420a-002a-47d5-be64-bfaa4cd11fca admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdda71044faf489ebe9ba5c4862b5768)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:15.583378 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d0420a-002a-47d5-be64-bfaa4cd11fca admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:15.589926 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:15.854572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c7d0420a-002a-47d5-be64-bfaa4cd11fca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdda71044faf489ebe9ba5c4862b5768'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:15.937074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c7d0420a-002a-47d5-be64-bfaa4cd11fca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdda71044faf489ebe9ba5c4862b5768'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:15.945232 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c7d0420a-002a-47d5-be64-bfaa4cd11fca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdda71044faf489ebe9ba5c4862b5768'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:15.948129 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c7d0420a-002a-47d5-be64-bfaa4cd11fca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdda71044faf489ebe9ba5c4862b5768'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:15.958945 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c7d0420a-002a-47d5-be64-bfaa4cd11fca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdda71044faf489ebe9ba5c4862b5768'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:15.965578 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1254/2501] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:39:15 2022] DELETE /identity/v3/users/fdda71044faf489ebe9ba5c4862b5768 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:16.651389 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01226c9a-3286-4496-bf65-61cd2e9ddb57 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:16.652847 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-01226c9a-3286-4496-bf65-61cd2e9ddb57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:16.687519 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01226c9a-3286-4496-bf65-61cd2e9ddb57 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:16.759799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce28b0e-6272-4f0c-8175-12efc222f47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:16.760793 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce28b0e-6272-4f0c-8175-12efc222f47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:16.761074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce28b0e-6272-4f0c-8175-12efc222f47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:16.761350 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce28b0e-6272-4f0c-8175-12efc222f47f admin admin] PATH_INFO: `/v3/projects/72187061e9ba4275b81c7ee42529a1ad` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:16.762589 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce28b0e-6272-4f0c-8175-12efc222f47f admin admin] RBAC: Authorizing `identity:delete_project(project_id=72187061e9ba4275b81c7ee42529a1ad)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:16.764749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce28b0e-6272-4f0c-8175-12efc222f47f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:16.918653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5ce28b0e-6272-4f0c-8175-12efc222f47f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72187061e9ba4275b81c7ee42529a1ad', 'request_id': 'req-5ce28b0e-6272-4f0c-8175-12efc222f47f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:17.045705 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5ce28b0e-6272-4f0c-8175-12efc222f47f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72187061e9ba4275b81c7ee42529a1ad', 'request_id': 'req-5ce28b0e-6272-4f0c-8175-12efc222f47f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:17.056123 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5ce28b0e-6272-4f0c-8175-12efc222f47f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72187061e9ba4275b81c7ee42529a1ad', 'request_id': 'req-5ce28b0e-6272-4f0c-8175-12efc222f47f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:17.066386 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:17.073636 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5ce28b0e-6272-4f0c-8175-12efc222f47f admin admin] The token cache is being invalidate because project 72187061e9ba4275b81c7ee42529a1ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:39:17.074247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5ce28b0e-6272-4f0c-8175-12efc222f47f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:17.075389 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5ce28b0e-6272-4f0c-8175-12efc222f47f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:17.078618 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1248/2502] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:39:16 2022] DELETE /identity/v3/projects/72187061e9ba4275b81c7ee42529a1ad => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:17.084029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97b29b-4951-4f2e-9081-bb4ac922334b None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:17.085364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97b29b-4951-4f2e-9081-bb4ac922334b None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:17.085364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97b29b-4951-4f2e-9081-bb4ac922334b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:17.103885 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-be97b29b-4951-4f2e-9081-bb4ac922334b None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:17.105695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-be97b29b-4951-4f2e-9081-bb4ac922334b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:17.171180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1255/2503] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:17.179060 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d05553-f6cf-4de7-8804-372120660383 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:17.179348 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d05553-f6cf-4de7-8804-372120660383 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:17.179639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d05553-f6cf-4de7-8804-372120660383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:17.199155 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-23d05553-f6cf-4de7-8804-372120660383 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:17.200371 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-23d05553-f6cf-4de7-8804-372120660383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:17.208749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1249/2504] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:17.213817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97b29b-4951-4f2e-9081-bb4ac922334b None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:17.223555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97b29b-4951-4f2e-9081-bb4ac922334b None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:17.228259 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb24823-b7d8-4129-953a-e6712307f6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:17.230506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb24823-b7d8-4129-953a-e6712307f6ab admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:17.230506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb24823-b7d8-4129-953a-e6712307f6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:17.230506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb24823-b7d8-4129-953a-e6712307f6ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:17.230852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb24823-b7d8-4129-953a-e6712307f6ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:39:17.231105 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb24823-b7d8-4129-953a-e6712307f6ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:17.232865 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb24823-b7d8-4129-953a-e6712307f6ab admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:17.248571 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1256/2505] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:39:17.253269 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d05553-f6cf-4de7-8804-372120660383 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:17.270016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d05553-f6cf-4de7-8804-372120660383 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:17.275208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587418a5-2f2a-4f84-9c61-8f596ec6552c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:17.276589 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587418a5-2f2a-4f84-9c61-8f596ec6552c admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:17.277057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587418a5-2f2a-4f84-9c61-8f596ec6552c admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:17.277625 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587418a5-2f2a-4f84-9c61-8f596ec6552c admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:17.278971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587418a5-2f2a-4f84-9c61-8f596ec6552c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:17.282595 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587418a5-2f2a-4f84-9c61-8f596ec6552c admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:17.392835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1250/2506] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:39:17 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:17.397164 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb24823-b7d8-4129-953a-e6712307f6ab admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:17.411485 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb24823-b7d8-4129-953a-e6712307f6ab admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:17.493993 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14df038-c491-4c3e-a73b-c13129a1bf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:17.494724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14df038-c491-4c3e-a73b-c13129a1bf22 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:17.495005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14df038-c491-4c3e-a73b-c13129a1bf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:17.495289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14df038-c491-4c3e-a73b-c13129a1bf22 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:17.495976 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14df038-c491-4c3e-a73b-c13129a1bf22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:17.498003 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14df038-c491-4c3e-a73b-c13129a1bf22 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:17.605454 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:17.610629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1257/2507] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:39:17 2022] POST /identity/v3/users => generated 283 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:17.615571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587418a5-2f2a-4f84-9c61-8f596ec6552c admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:17.627915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587418a5-2f2a-4f84-9c61-8f596ec6552c admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:17.631493 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759721da-c9c1-4c3d-bad4-7c41067cd687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:17.632309 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759721da-c9c1-4c3d-bad4-7c41067cd687 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:17.632694 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759721da-c9c1-4c3d-bad4-7c41067cd687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:17.633224 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759721da-c9c1-4c3d-bad4-7c41067cd687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:17.634087 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759721da-c9c1-4c3d-bad4-7c41067cd687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:17.636435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759721da-c9c1-4c3d-bad4-7c41067cd687 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:17.650197 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1251/2508] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:17.658734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14df038-c491-4c3e-a73b-c13129a1bf22 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:17.671446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14df038-c491-4c3e-a73b-c13129a1bf22 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:17.676245 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa414d-aa11-4d12-b509-5883ac67f994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:17.677077 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa414d-aa11-4d12-b509-5883ac67f994 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:17.677415 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa414d-aa11-4d12-b509-5883ac67f994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:17.678046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa414d-aa11-4d12-b509-5883ac67f994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:17.678854 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa414d-aa11-4d12-b509-5883ac67f994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:17.681289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa414d-aa11-4d12-b509-5883ac67f994 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:17.701381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1258/2509] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:17.704518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759721da-c9c1-4c3d-bad4-7c41067cd687 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:17.717677 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759721da-c9c1-4c3d-bad4-7c41067cd687 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:17.728332 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e021e-63ad-4396-969a-fd9c92d44456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:17.729289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e021e-63ad-4396-969a-fd9c92d44456 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:17.729571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e021e-63ad-4396-969a-fd9c92d44456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:17.738979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e021e-63ad-4396-969a-fd9c92d44456 admin admin] PATH_INFO: `/v3/projects/ab7b7ffcecfe494bb04bad88878ba894/users/d9f38191c164488f8e084a20820a37ee/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:17.745381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e021e-63ad-4396-969a-fd9c92d44456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7b7ffcecfe494bb04bad88878ba894, user_id=d9f38191c164488f8e084a20820a37ee, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:17.748152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e021e-63ad-4396-969a-fd9c92d44456 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:17.911723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1252/2510] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:17 2022] PUT /identity/v3/projects/ab7b7ffcecfe494bb04bad88878ba894/users/d9f38191c164488f8e084a20820a37ee/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:17.922485 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717dd62-0429-4d7e-be9a-171ef4cb5873 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:17.923082 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717dd62-0429-4d7e-be9a-171ef4cb5873 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:17.923365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717dd62-0429-4d7e-be9a-171ef4cb5873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:17.953004 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-2717dd62-0429-4d7e-be9a-171ef4cb5873 None None] MFA Rules not processed for user `d9f38191c164488f8e084a20820a37ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:17.954278 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2717dd62-0429-4d7e-be9a-171ef4cb5873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:18.055616 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1259/2511] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:17 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:18.060778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e021e-63ad-4396-969a-fd9c92d44456 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:18.171487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e021e-63ad-4396-969a-fd9c92d44456 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:18.177749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7ba20-6023-4eae-b9cc-da8d6000f8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:18.179564 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7ba20-6023-4eae-b9cc-da8d6000f8e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:18.180010 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7ba20-6023-4eae-b9cc-da8d6000f8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:18.180504 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7ba20-6023-4eae-b9cc-da8d6000f8e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:18.181549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7ba20-6023-4eae-b9cc-da8d6000f8e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:18.185302 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7ba20-6023-4eae-b9cc-da8d6000f8e7 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:18.323016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1253/2512] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:39:18 2022] POST /identity/v3/projects => generated 363 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:18.327261 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717dd62-0429-4d7e-be9a-171ef4cb5873 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:18.341076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717dd62-0429-4d7e-be9a-171ef4cb5873 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:18.431494 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb76b3-eb00-4ea0-bdfd-f3f99e9f968b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:18.432479 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb76b3-eb00-4ea0-bdfd-f3f99e9f968b admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:18.433695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb76b3-eb00-4ea0-bdfd-f3f99e9f968b admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:18.434142 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb76b3-eb00-4ea0-bdfd-f3f99e9f968b admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:18.439207 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeb76b3-eb00-4ea0-bdfd-f3f99e9f968b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:18.441272 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeb76b3-eb00-4ea0-bdfd-f3f99e9f968b admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:18.604572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1260/2513] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:39:18 2022] POST /identity/v3/users => generated 288 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:39:18.605886 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7ba20-6023-4eae-b9cc-da8d6000f8e7 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:18.625673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7ba20-6023-4eae-b9cc-da8d6000f8e7 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:18.630528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a447c9b-e0ee-41b3-a81d-65ef3ffa14c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:18.631214 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a447c9b-e0ee-41b3-a81d-65ef3ffa14c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:18.631500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a447c9b-e0ee-41b3-a81d-65ef3ffa14c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:18.631779 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a447c9b-e0ee-41b3-a81d-65ef3ffa14c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:18.632546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a447c9b-e0ee-41b3-a81d-65ef3ffa14c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:18.634965 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a447c9b-e0ee-41b3-a81d-65ef3ffa14c4 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:18.655190 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1254/2514] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:18.656208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb76b3-eb00-4ea0-bdfd-f3f99e9f968b admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:18.667255 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb76b3-eb00-4ea0-bdfd-f3f99e9f968b admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:18.672338 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e9e3d-c8d2-4b87-9859-ea0d426eed44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:18.673479 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e9e3d-c8d2-4b87-9859-ea0d426eed44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:18.673789 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e9e3d-c8d2-4b87-9859-ea0d426eed44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:18.674056 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e9e3d-c8d2-4b87-9859-ea0d426eed44 admin admin] PATH_INFO: `/v3/projects/41e2ff698783414fa7b43920f124d4dc/users/2c3b5f8db6e243bb8e2eaac1cf591102/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:18.699341 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e9e3d-c8d2-4b87-9859-ea0d426eed44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41e2ff698783414fa7b43920f124d4dc, user_id=2c3b5f8db6e243bb8e2eaac1cf591102, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:18.702165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e9e3d-c8d2-4b87-9859-ea0d426eed44 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:18.906065 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1261/2515] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:18 2022] PUT /identity/v3/projects/41e2ff698783414fa7b43920f124d4dc/users/2c3b5f8db6e243bb8e2eaac1cf591102/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:18.913151 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a447c9b-e0ee-41b3-a81d-65ef3ffa14c4 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:18.924334 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a447c9b-e0ee-41b3-a81d-65ef3ffa14c4 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:18.990888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28b66f-4401-4beb-b1c6-026d2fcc72af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:18.991547 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28b66f-4401-4beb-b1c6-026d2fcc72af admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:18.991836 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28b66f-4401-4beb-b1c6-026d2fcc72af admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:18.992115 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28b66f-4401-4beb-b1c6-026d2fcc72af admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:18.992851 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28b66f-4401-4beb-b1c6-026d2fcc72af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:18.994539 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28b66f-4401-4beb-b1c6-026d2fcc72af admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:19.004967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1255/2516] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:19.009433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e9e3d-c8d2-4b87-9859-ea0d426eed44 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:19.025436 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e9e3d-c8d2-4b87-9859-ea0d426eed44 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:19.031115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bc533-a34c-48b1-b459-c8b787cedd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:19.032045 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bc533-a34c-48b1-b459-c8b787cedd47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:19.032326 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bc533-a34c-48b1-b459-c8b787cedd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:19.032616 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bc533-a34c-48b1-b459-c8b787cedd47 admin admin] PATH_INFO: `/v3/projects/41e2ff698783414fa7b43920f124d4dc/users/2c3b5f8db6e243bb8e2eaac1cf591102/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:19.035119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74bc533-a34c-48b1-b459-c8b787cedd47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41e2ff698783414fa7b43920f124d4dc, user_id=2c3b5f8db6e243bb8e2eaac1cf591102, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:19.038826 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74bc533-a34c-48b1-b459-c8b787cedd47 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:19.052574 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1262/2517] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:19 2022] PUT /identity/v3/projects/41e2ff698783414fa7b43920f124d4dc/users/2c3b5f8db6e243bb8e2eaac1cf591102/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:19.064895 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64607b-2f40-4813-9700-34df5e062e70 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:19.065180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64607b-2f40-4813-9700-34df5e062e70 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:19.065462 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64607b-2f40-4813-9700-34df5e062e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:19.079069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:19.106837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-2c64607b-2f40-4813-9700-34df5e062e70 None None] MFA Rules not processed for user `2c3b5f8db6e243bb8e2eaac1cf591102`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:19.108425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-2c64607b-2f40-4813-9700-34df5e062e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:19.192973 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1256/2518] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:19 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:19.206912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bc533-a34c-48b1-b459-c8b787cedd47 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:19.208620 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c74bc533-a34c-48b1-b459-c8b787cedd47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:19.251090 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:19.286491 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bc533-a34c-48b1-b459-c8b787cedd47 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:19.289960 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd886fa3-1e7d-4d78-9eff-11ec5c20a41d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:19.290745 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd886fa3-1e7d-4d78-9eff-11ec5c20a41d service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:19.291122 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd886fa3-1e7d-4d78-9eff-11ec5c20a41d service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:19.291475 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd886fa3-1e7d-4d78-9eff-11ec5c20a41d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:19.304019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd886fa3-1e7d-4d78-9eff-11ec5c20a41d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:19.305606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd886fa3-1e7d-4d78-9eff-11ec5c20a41d service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:19.318979 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1263/2519] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:19 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:39:20.922690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64607b-2f40-4813-9700-34df5e062e70 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:21.014436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64607b-2f40-4813-9700-34df5e062e70 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:21.019433 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073e3fb-ad5b-4bc8-b69e-80609759dd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:21.022814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073e3fb-ad5b-4bc8-b69e-80609759dd94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:21.022814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073e3fb-ad5b-4bc8-b69e-80609759dd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:21.022814 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073e3fb-ad5b-4bc8-b69e-80609759dd94 admin admin] PATH_INFO: `/v3/users/d9f38191c164488f8e084a20820a37ee` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:21.027169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073e3fb-ad5b-4bc8-b69e-80609759dd94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9f38191c164488f8e084a20820a37ee)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:21.029112 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073e3fb-ad5b-4bc8-b69e-80609759dd94 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:21.254164 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-6073e3fb-ad5b-4bc8-b69e-80609759dd94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9f38191c164488f8e084a20820a37ee'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:21.377406 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-6073e3fb-ad5b-4bc8-b69e-80609759dd94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9f38191c164488f8e084a20820a37ee'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:21.378014 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-6073e3fb-ad5b-4bc8-b69e-80609759dd94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9f38191c164488f8e084a20820a37ee'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:21.390826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-6073e3fb-ad5b-4bc8-b69e-80609759dd94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9f38191c164488f8e084a20820a37ee'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:21.398888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-6073e3fb-ad5b-4bc8-b69e-80609759dd94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9f38191c164488f8e084a20820a37ee'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:21.405325 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1257/2520] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:39:20 2022] DELETE /identity/v3/users/d9f38191c164488f8e084a20820a37ee => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:21.410027 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e3449-dc05-4883-b49d-17af40c4ac4e None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:21.410627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e3449-dc05-4883-b49d-17af40c4ac4e None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:21.410744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e3449-dc05-4883-b49d-17af40c4ac4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:21.429811 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-d45e3449-dc05-4883-b49d-17af40c4ac4e None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:21.431061 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d45e3449-dc05-4883-b49d-17af40c4ac4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:21.505840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1264/2521] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:21.517184 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073e3fb-ad5b-4bc8-b69e-80609759dd94 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:21.518533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-6073e3fb-ad5b-4bc8-b69e-80609759dd94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:21.532909 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:21.576156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073e3fb-ad5b-4bc8-b69e-80609759dd94 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:21.579755 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438225d-7aad-4a2b-9798-e71c5ac3c609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:21.580655 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438225d-7aad-4a2b-9798-e71c5ac3c609 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:21.580946 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438225d-7aad-4a2b-9798-e71c5ac3c609 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:21.581222 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438225d-7aad-4a2b-9798-e71c5ac3c609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:21.591425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438225d-7aad-4a2b-9798-e71c5ac3c609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:21.601390 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438225d-7aad-4a2b-9798-e71c5ac3c609 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:21.602493 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1258/2522] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:22.045115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e3449-dc05-4883-b49d-17af40c4ac4e None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:22.046685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d45e3449-dc05-4883-b49d-17af40c4ac4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:22.059051 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e3449-dc05-4883-b49d-17af40c4ac4e None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:22.064073 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b520c-8293-4ec7-b54f-74e73a30690f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:22.064982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b520c-8293-4ec7-b54f-74e73a30690f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:22.065730 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b520c-8293-4ec7-b54f-74e73a30690f admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:22.065730 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b520c-8293-4ec7-b54f-74e73a30690f admin admin] PATH_INFO: `/v3/projects/ab7b7ffcecfe494bb04bad88878ba894` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:22.066915 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b520c-8293-4ec7-b54f-74e73a30690f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab7b7ffcecfe494bb04bad88878ba894)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:22.068833 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b520c-8293-4ec7-b54f-74e73a30690f admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:22.209840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0f9b520c-8293-4ec7-b54f-74e73a30690f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab7b7ffcecfe494bb04bad88878ba894', 'request_id': 'req-0f9b520c-8293-4ec7-b54f-74e73a30690f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:22.300635 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0f9b520c-8293-4ec7-b54f-74e73a30690f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab7b7ffcecfe494bb04bad88878ba894', 'request_id': 'req-0f9b520c-8293-4ec7-b54f-74e73a30690f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:22.308195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0f9b520c-8293-4ec7-b54f-74e73a30690f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab7b7ffcecfe494bb04bad88878ba894', 'request_id': 'req-0f9b520c-8293-4ec7-b54f-74e73a30690f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:22.320710 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0f9b520c-8293-4ec7-b54f-74e73a30690f admin admin] The token cache is being invalidate because project ab7b7ffcecfe494bb04bad88878ba894 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:39:22.321002 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0f9b520c-8293-4ec7-b54f-74e73a30690f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:22.321690 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-0f9b520c-8293-4ec7-b54f-74e73a30690f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:22.323915 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1265/2523] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:39:22 2022] DELETE /identity/v3/projects/ab7b7ffcecfe494bb04bad88878ba894 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:22.327964 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438225d-7aad-4a2b-9798-e71c5ac3c609 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:22.338644 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438225d-7aad-4a2b-9798-e71c5ac3c609 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:22.397807 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ffd4b-8fd5-462d-98c1-3868e2e764e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:22.398723 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ffd4b-8fd5-462d-98c1-3868e2e764e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:22.399018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ffd4b-8fd5-462d-98c1-3868e2e764e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:22.399298 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ffd4b-8fd5-462d-98c1-3868e2e764e7 admin admin] PATH_INFO: `/v3/users/2c3b5f8db6e243bb8e2eaac1cf591102` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:22.400533 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ffd4b-8fd5-462d-98c1-3868e2e764e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c3b5f8db6e243bb8e2eaac1cf591102)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:22.402659 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ffd4b-8fd5-462d-98c1-3868e2e764e7 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:22.761032 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-f61ffd4b-8fd5-462d-98c1-3868e2e764e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c3b5f8db6e243bb8e2eaac1cf591102'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:22.840830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-f61ffd4b-8fd5-462d-98c1-3868e2e764e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c3b5f8db6e243bb8e2eaac1cf591102'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:22.841426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-f61ffd4b-8fd5-462d-98c1-3868e2e764e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c3b5f8db6e243bb8e2eaac1cf591102'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:22.855180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-f61ffd4b-8fd5-462d-98c1-3868e2e764e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c3b5f8db6e243bb8e2eaac1cf591102'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:22.863852 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-f61ffd4b-8fd5-462d-98c1-3868e2e764e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c3b5f8db6e243bb8e2eaac1cf591102'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:22.871541 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1259/2524] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:39:22 2022] DELETE /identity/v3/users/2c3b5f8db6e243bb8e2eaac1cf591102 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:23.470162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b520c-8293-4ec7-b54f-74e73a30690f admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:23.471836 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0f9b520c-8293-4ec7-b54f-74e73a30690f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:23.500182 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b520c-8293-4ec7-b54f-74e73a30690f admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:23.556839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:23.563109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67dda6-99c7-440e-b3d0-8714efc95aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:23.564048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67dda6-99c7-440e-b3d0-8714efc95aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:23.564332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67dda6-99c7-440e-b3d0-8714efc95aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:23.564603 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67dda6-99c7-440e-b3d0-8714efc95aeb admin admin] PATH_INFO: `/v3/projects/41e2ff698783414fa7b43920f124d4dc` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:23.565846 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67dda6-99c7-440e-b3d0-8714efc95aeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=41e2ff698783414fa7b43920f124d4dc)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:23.568040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67dda6-99c7-440e-b3d0-8714efc95aeb admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:23.773181 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7d67dda6-99c7-440e-b3d0-8714efc95aeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41e2ff698783414fa7b43920f124d4dc', 'request_id': 'req-7d67dda6-99c7-440e-b3d0-8714efc95aeb'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:23.952602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7d67dda6-99c7-440e-b3d0-8714efc95aeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41e2ff698783414fa7b43920f124d4dc', 'request_id': 'req-7d67dda6-99c7-440e-b3d0-8714efc95aeb'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:23.962917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7d67dda6-99c7-440e-b3d0-8714efc95aeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41e2ff698783414fa7b43920f124d4dc', 'request_id': 'req-7d67dda6-99c7-440e-b3d0-8714efc95aeb'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:23.971298 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7d67dda6-99c7-440e-b3d0-8714efc95aeb admin admin] The token cache is being invalidate because project 41e2ff698783414fa7b43920f124d4dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:39:23.971744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7d67dda6-99c7-440e-b3d0-8714efc95aeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:23.973114 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7d67dda6-99c7-440e-b3d0-8714efc95aeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:23.974786 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1266/2525] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:39:23 2022] DELETE /identity/v3/projects/41e2ff698783414fa7b43920f124d4dc => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:23.981678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cc7b2-1627-4720-9ef6-c57e2b283d4f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:23.982196 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cc7b2-1627-4720-9ef6-c57e2b283d4f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:23.982371 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cc7b2-1627-4720-9ef6-c57e2b283d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:24.002848 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-072cc7b2-1627-4720-9ef6-c57e2b283d4f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:24.003819 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-072cc7b2-1627-4720-9ef6-c57e2b283d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:24.049971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:24.089733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1260/2526] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:24.097897 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89a006-a022-4ff0-a34d-6f95729fdc9e None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:24.098366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89a006-a022-4ff0-a34d-6f95729fdc9e None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:24.098778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89a006-a022-4ff0-a34d-6f95729fdc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:24.130533 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6b89a006-a022-4ff0-a34d-6f95729fdc9e None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:24.131765 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6b89a006-a022-4ff0-a34d-6f95729fdc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:24.140439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1267/2527] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:24.144715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cc7b2-1627-4720-9ef6-c57e2b283d4f None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:24.154858 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cc7b2-1627-4720-9ef6-c57e2b283d4f None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:24.166575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274f5ec-6c20-4f76-8de6-94970435bff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:24.166575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274f5ec-6c20-4f76-8de6-94970435bff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:24.166575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274f5ec-6c20-4f76-8de6-94970435bff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:24.166575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274f5ec-6c20-4f76-8de6-94970435bff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:24.167035 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274f5ec-6c20-4f76-8de6-94970435bff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:39:24.167035 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274f5ec-6c20-4f76-8de6-94970435bff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:24.167035 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274f5ec-6c20-4f76-8de6-94970435bff7 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:24.178302 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1261/2528] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:39:24.183493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89a006-a022-4ff0-a34d-6f95729fdc9e None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:24.194341 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89a006-a022-4ff0-a34d-6f95729fdc9e None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:24.199235 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f8e42-567a-4958-9aa6-14f36890782b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:24.199935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f8e42-567a-4958-9aa6-14f36890782b admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:24.200213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f8e42-567a-4958-9aa6-14f36890782b admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:24.200694 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f8e42-567a-4958-9aa6-14f36890782b admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:24.201527 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f8e42-567a-4958-9aa6-14f36890782b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:24.203611 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f8e42-567a-4958-9aa6-14f36890782b admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:24.424171 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1268/2529] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:39:24 2022] POST /identity/v3/projects => generated 407 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:24.428535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274f5ec-6c20-4f76-8de6-94970435bff7 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:24.439841 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274f5ec-6c20-4f76-8de6-94970435bff7 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:24.504194 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed38635-14be-4afb-9c6f-c983f665f6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:24.505124 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38635-14be-4afb-9c6f-c983f665f6e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:24.505275 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38635-14be-4afb-9c6f-c983f665f6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:24.505503 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38635-14be-4afb-9c6f-c983f665f6e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:24.506251 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed38635-14be-4afb-9c6f-c983f665f6e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:24.508305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed38635-14be-4afb-9c6f-c983f665f6e7 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:24.659211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1262/2530] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:39:24 2022] POST /identity/v3/users => generated 304 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:24.663556 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f8e42-567a-4958-9aa6-14f36890782b admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:24.674531 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f8e42-567a-4958-9aa6-14f36890782b admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:24.679136 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd94786-b687-4262-8c6a-fff4cfd9e8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:24.679831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd94786-b687-4262-8c6a-fff4cfd9e8af admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:24.680103 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd94786-b687-4262-8c6a-fff4cfd9e8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:24.680382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd94786-b687-4262-8c6a-fff4cfd9e8af admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:24.681109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd94786-b687-4262-8c6a-fff4cfd9e8af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:24.682938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd94786-b687-4262-8c6a-fff4cfd9e8af admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:24.694187 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1269/2531] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:24.698664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed38635-14be-4afb-9c6f-c983f665f6e7 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:24.708910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed38635-14be-4afb-9c6f-c983f665f6e7 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:24.714128 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf33a1-5415-4b0e-b8d9-296d8f804eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:24.714811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf33a1-5415-4b0e-b8d9-296d8f804eba admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:24.715094 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf33a1-5415-4b0e-b8d9-296d8f804eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:24.715368 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf33a1-5415-4b0e-b8d9-296d8f804eba admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:24.716090 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf33a1-5415-4b0e-b8d9-296d8f804eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:24.717653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf33a1-5415-4b0e-b8d9-296d8f804eba admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:24.727621 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1263/2532] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:24.732504 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd94786-b687-4262-8c6a-fff4cfd9e8af admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:24.742650 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd94786-b687-4262-8c6a-fff4cfd9e8af admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:24.747399 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f4f32-d85e-4f6a-b2b3-35f9fa2c2480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:24.748293 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f4f32-d85e-4f6a-b2b3-35f9fa2c2480 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:24.748578 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f4f32-d85e-4f6a-b2b3-35f9fa2c2480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:24.748853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f4f32-d85e-4f6a-b2b3-35f9fa2c2480 admin admin] PATH_INFO: `/v3/projects/a07ead20767d4875b07e23d17f35bd78/users/4593797ea1b6463f8b8ac7a37b01dce6/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:24.764951 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39f4f32-d85e-4f6a-b2b3-35f9fa2c2480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07ead20767d4875b07e23d17f35bd78, user_id=4593797ea1b6463f8b8ac7a37b01dce6, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:24.767663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39f4f32-d85e-4f6a-b2b3-35f9fa2c2480 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:24.898623 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1270/2533] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:24 2022] PUT /identity/v3/projects/a07ead20767d4875b07e23d17f35bd78/users/4593797ea1b6463f8b8ac7a37b01dce6/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:24.909820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc163442-30c8-4eee-b597-c3a3bceb4851 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:24.910099 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc163442-30c8-4eee-b597-c3a3bceb4851 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:24.910404 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc163442-30c8-4eee-b597-c3a3bceb4851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:24.943022 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-bc163442-30c8-4eee-b597-c3a3bceb4851 None None] MFA Rules not processed for user `4593797ea1b6463f8b8ac7a37b01dce6`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:24.947885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-bc163442-30c8-4eee-b597-c3a3bceb4851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:25.018001 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1264/2534] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:24 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:25.022587 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f4f32-d85e-4f6a-b2b3-35f9fa2c2480 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:25.091633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f4f32-d85e-4f6a-b2b3-35f9fa2c2480 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:25.097225 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166f900-ea3a-40c9-bc6c-a77219e097a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:25.099081 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166f900-ea3a-40c9-bc6c-a77219e097a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:25.099567 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166f900-ea3a-40c9-bc6c-a77219e097a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:25.100516 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166f900-ea3a-40c9-bc6c-a77219e097a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:25.101555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166f900-ea3a-40c9-bc6c-a77219e097a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:25.105317 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166f900-ea3a-40c9-bc6c-a77219e097a0 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:26.418413 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1271/2535] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:39:25 2022] POST /identity/v3/projects => generated 407 bytes in 1396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:26.422708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc163442-30c8-4eee-b597-c3a3bceb4851 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:26.434511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc163442-30c8-4eee-b597-c3a3bceb4851 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:26.503530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019cd5c-b770-4506-bf52-34ca257f5d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:26.504231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019cd5c-b770-4506-bf52-34ca257f5d9d admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:26.504517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019cd5c-b770-4506-bf52-34ca257f5d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:26.504981 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019cd5c-b770-4506-bf52-34ca257f5d9d admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:26.506312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019cd5c-b770-4506-bf52-34ca257f5d9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:26.508407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019cd5c-b770-4506-bf52-34ca257f5d9d admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:26.759444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1265/2536] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:39:26 2022] POST /identity/v3/users => generated 310 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:26.763927 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166f900-ea3a-40c9-bc6c-a77219e097a0 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:26.777223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166f900-ea3a-40c9-bc6c-a77219e097a0 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:26.784611 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc9d0c-79b2-4c7f-a2a8-0ca94acffcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:26.816867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc9d0c-79b2-4c7f-a2a8-0ca94acffcc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:26.816867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc9d0c-79b2-4c7f-a2a8-0ca94acffcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:26.816867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc9d0c-79b2-4c7f-a2a8-0ca94acffcc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:26.816867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc9d0c-79b2-4c7f-a2a8-0ca94acffcc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:26.816867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc9d0c-79b2-4c7f-a2a8-0ca94acffcc7 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:26.816867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:26.817451 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1272/2537] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:26.820076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019cd5c-b770-4506-bf52-34ca257f5d9d admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:26.831697 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019cd5c-b770-4506-bf52-34ca257f5d9d admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:26.837174 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c549d7-f13d-4d83-a8b1-7017b94127f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:26.839166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c549d7-f13d-4d83-a8b1-7017b94127f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:26.839166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c549d7-f13d-4d83-a8b1-7017b94127f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:26.839166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c549d7-f13d-4d83-a8b1-7017b94127f4 admin admin] PATH_INFO: `/v3/projects/2dba3c55e04f4a209edbb7a20a485428/users/46cadf8ec0ba43e290d34f8a5521a22f/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:26.865984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c549d7-f13d-4d83-a8b1-7017b94127f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dba3c55e04f4a209edbb7a20a485428, user_id=46cadf8ec0ba43e290d34f8a5521a22f, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:26.871324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c549d7-f13d-4d83-a8b1-7017b94127f4 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:27.050464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1266/2538] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:26 2022] PUT /identity/v3/projects/2dba3c55e04f4a209edbb7a20a485428/users/46cadf8ec0ba43e290d34f8a5521a22f/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:27.054920 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc9d0c-79b2-4c7f-a2a8-0ca94acffcc7 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:27.066635 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc9d0c-79b2-4c7f-a2a8-0ca94acffcc7 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:27.135138 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9dd3db-4a9e-410a-8a1b-91bd850825e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:27.136037 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9dd3db-4a9e-410a-8a1b-91bd850825e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:27.136334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9dd3db-4a9e-410a-8a1b-91bd850825e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:27.136651 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9dd3db-4a9e-410a-8a1b-91bd850825e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:27.137456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9dd3db-4a9e-410a-8a1b-91bd850825e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:27.139152 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9dd3db-4a9e-410a-8a1b-91bd850825e8 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:27.151108 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1273/2539] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:27.169678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c549d7-f13d-4d83-a8b1-7017b94127f4 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:27.180072 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c549d7-f13d-4d83-a8b1-7017b94127f4 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:27.185524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f976312c-1654-4d6a-9f80-0b3567ee9764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:27.187208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976312c-1654-4d6a-9f80-0b3567ee9764 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:27.187208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976312c-1654-4d6a-9f80-0b3567ee9764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:27.187208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976312c-1654-4d6a-9f80-0b3567ee9764 admin admin] PATH_INFO: `/v3/projects/2dba3c55e04f4a209edbb7a20a485428/users/46cadf8ec0ba43e290d34f8a5521a22f/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:27.189335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f976312c-1654-4d6a-9f80-0b3567ee9764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dba3c55e04f4a209edbb7a20a485428, user_id=46cadf8ec0ba43e290d34f8a5521a22f, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:27.192036 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f976312c-1654-4d6a-9f80-0b3567ee9764 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:27.202785 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1267/2540] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:27 2022] PUT /identity/v3/projects/2dba3c55e04f4a209edbb7a20a485428/users/46cadf8ec0ba43e290d34f8a5521a22f/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:27.224787 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff92de1-1096-401c-ba6e-5da7b91b21a2 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:27.224787 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff92de1-1096-401c-ba6e-5da7b91b21a2 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:27.224787 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff92de1-1096-401c-ba6e-5da7b91b21a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:27.254493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-9ff92de1-1096-401c-ba6e-5da7b91b21a2 None None] MFA Rules not processed for user `46cadf8ec0ba43e290d34f8a5521a22f`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:27.255731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9ff92de1-1096-401c-ba6e-5da7b91b21a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:27.325355 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1274/2541] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:27 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:39:27.337319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f976312c-1654-4d6a-9f80-0b3567ee9764 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:27.345135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f976312c-1654-4d6a-9f80-0b3567ee9764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:27.411860 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f976312c-1654-4d6a-9f80-0b3567ee9764 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:27.416617 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970121b-291d-413a-bba5-4d1419c82dd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:27.417470 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970121b-291d-413a-bba5-4d1419c82dd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:27.418351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970121b-291d-413a-bba5-4d1419c82dd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:27.418351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970121b-291d-413a-bba5-4d1419c82dd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:27.429599 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970121b-291d-413a-bba5-4d1419c82dd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:27.432466 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970121b-291d-413a-bba5-4d1419c82dd0 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:27.443737 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1268/2542] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:27 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:27.801121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:28.386282 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f0d9c-4883-468a-a736-53ea16e2ba8b None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:28.386861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f0d9c-4883-468a-a736-53ea16e2ba8b None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:28.387079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f0d9c-4883-468a-a736-53ea16e2ba8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:28.405406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-1e3f0d9c-4883-468a-a736-53ea16e2ba8b None None] MFA Rules not processed for user `0235b8fe92554131ba30e1d18f0f2d86`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:28.406656 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1e3f0d9c-4883-468a-a736-53ea16e2ba8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:28.411769 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970121b-291d-413a-bba5-4d1419c82dd0 service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:28.420289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970121b-291d-413a-bba5-4d1419c82dd0 service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:28.424249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce585f2f-8dc1-4d1d-b100-066712ba5aff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:28.424897 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce585f2f-8dc1-4d1d-b100-066712ba5aff service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:28.425179 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce585f2f-8dc1-4d1d-b100-066712ba5aff service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:28.425456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce585f2f-8dc1-4d1d-b100-066712ba5aff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:28.463444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1275/2543] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:28.488758 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce585f2f-8dc1-4d1d-b100-066712ba5aff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:28.505678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce585f2f-8dc1-4d1d-b100-066712ba5aff service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:28.505678 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1269/2544] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:28 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:29.662429 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc089af1-6229-40bf-aaab-63d8d6772c8b None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:29.662872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc089af1-6229-40bf-aaab-63d8d6772c8b None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:29.663096 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc089af1-6229-40bf-aaab-63d8d6772c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:29.680997 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-fc089af1-6229-40bf-aaab-63d8d6772c8b None None] MFA Rules not processed for user `4e75a82cc7e94702ba0080fc65d6eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:29.682362 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-fc089af1-6229-40bf-aaab-63d8d6772c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:29.740418 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1276/2545] 10.222.0.78 () {58 vars in 967 bytes} [Wed Jul 20 15:39:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:29.766895 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce585f2f-8dc1-4d1d-b100-066712ba5aff service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:29.768366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ce585f2f-8dc1-4d1d-b100-066712ba5aff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:29.843974 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce585f2f-8dc1-4d1d-b100-066712ba5aff service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:29.849137 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3dd9f5-57a5-4ff6-b455-b2bc284355ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0d719c0eb04a3b8a228c6ce70fdf98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:29.849828 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3dd9f5-57a5-4ff6-b455-b2bc284355ea service swift] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:29.850116 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3dd9f5-57a5-4ff6-b455-b2bc284355ea service swift] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:29.854294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3dd9f5-57a5-4ff6-b455-b2bc284355ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:29.859205 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3dd9f5-57a5-4ff6-b455-b2bc284355ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:29.860799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3dd9f5-57a5-4ff6-b455-b2bc284355ea service swift] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:29.869690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1270/2546] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Jul 20 15:39:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:39:30.577025 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:30.861694 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:40.509811 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc089af1-6229-40bf-aaab-63d8d6772c8b None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:40.621219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc089af1-6229-40bf-aaab-63d8d6772c8b None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:40.626026 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae757ece-0a5c-480b-90a1-3093f10181f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:40.627000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae757ece-0a5c-480b-90a1-3093f10181f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:40.627288 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae757ece-0a5c-480b-90a1-3093f10181f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:40.627565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae757ece-0a5c-480b-90a1-3093f10181f0 admin admin] PATH_INFO: `/v3/users/4593797ea1b6463f8b8ac7a37b01dce6` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:40.628757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae757ece-0a5c-480b-90a1-3093f10181f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4593797ea1b6463f8b8ac7a37b01dce6)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:40.630749 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae757ece-0a5c-480b-90a1-3093f10181f0 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:40.853992 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ae757ece-0a5c-480b-90a1-3093f10181f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4593797ea1b6463f8b8ac7a37b01dce6'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:40.953687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ae757ece-0a5c-480b-90a1-3093f10181f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4593797ea1b6463f8b8ac7a37b01dce6'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:40.961742 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ae757ece-0a5c-480b-90a1-3093f10181f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4593797ea1b6463f8b8ac7a37b01dce6'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:40.962976 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ae757ece-0a5c-480b-90a1-3093f10181f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4593797ea1b6463f8b8ac7a37b01dce6'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:40.979023 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ae757ece-0a5c-480b-90a1-3093f10181f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4593797ea1b6463f8b8ac7a37b01dce6'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:40.994297 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1277/2547] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:39:40 2022] DELETE /identity/v3/users/4593797ea1b6463f8b8ac7a37b01dce6 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:40.998915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72007d25-ec13-4d24-80b1-628d2ce2f672 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:40.999481 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72007d25-ec13-4d24-80b1-628d2ce2f672 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:40.999481 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72007d25-ec13-4d24-80b1-628d2ce2f672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:41.052240 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-72007d25-ec13-4d24-80b1-628d2ce2f672 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:41.053469 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-72007d25-ec13-4d24-80b1-628d2ce2f672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:41.116334 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1271/2548] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:41.127070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae757ece-0a5c-480b-90a1-3093f10181f0 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:41.128464 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ae757ece-0a5c-480b-90a1-3093f10181f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:41.136783 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:41.210372 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae757ece-0a5c-480b-90a1-3093f10181f0 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:41.217097 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ac034-edda-4cc4-9b83-f23dfcf04926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:41.223366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ac034-edda-4cc4-9b83-f23dfcf04926 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:41.223366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ac034-edda-4cc4-9b83-f23dfcf04926 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:41.223366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ac034-edda-4cc4-9b83-f23dfcf04926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:41.231913 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ac034-edda-4cc4-9b83-f23dfcf04926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:41.233576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ac034-edda-4cc4-9b83-f23dfcf04926 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:41.245621 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1278/2549] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:41.899485 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72007d25-ec13-4d24-80b1-628d2ce2f672 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:41.901751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-72007d25-ec13-4d24-80b1-628d2ce2f672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:41.921793 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72007d25-ec13-4d24-80b1-628d2ce2f672 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:41.924458 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f9fe7-52ee-4242-9d5d-c59fac1be563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:41.926153 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f9fe7-52ee-4242-9d5d-c59fac1be563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:41.926772 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f9fe7-52ee-4242-9d5d-c59fac1be563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:41.927305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f9fe7-52ee-4242-9d5d-c59fac1be563 admin admin] PATH_INFO: `/v3/projects/a07ead20767d4875b07e23d17f35bd78` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:41.929817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f9fe7-52ee-4242-9d5d-c59fac1be563 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a07ead20767d4875b07e23d17f35bd78)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:41.931505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f9fe7-52ee-4242-9d5d-c59fac1be563 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:42.086885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-de5f9fe7-52ee-4242-9d5d-c59fac1be563 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a07ead20767d4875b07e23d17f35bd78', 'request_id': 'req-de5f9fe7-52ee-4242-9d5d-c59fac1be563'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:42.170703 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-de5f9fe7-52ee-4242-9d5d-c59fac1be563 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a07ead20767d4875b07e23d17f35bd78', 'request_id': 'req-de5f9fe7-52ee-4242-9d5d-c59fac1be563'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:42.177863 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-de5f9fe7-52ee-4242-9d5d-c59fac1be563 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a07ead20767d4875b07e23d17f35bd78', 'request_id': 'req-de5f9fe7-52ee-4242-9d5d-c59fac1be563'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:42.193810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-de5f9fe7-52ee-4242-9d5d-c59fac1be563 admin admin] The token cache is being invalidate because project a07ead20767d4875b07e23d17f35bd78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:39:42.194720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-de5f9fe7-52ee-4242-9d5d-c59fac1be563 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:42.195988 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-de5f9fe7-52ee-4242-9d5d-c59fac1be563 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:42.197608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1272/2550] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:39:41 2022] DELETE /identity/v3/projects/a07ead20767d4875b07e23d17f35bd78 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:42.201864 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ac034-edda-4cc4-9b83-f23dfcf04926 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:42.212321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ac034-edda-4cc4-9b83-f23dfcf04926 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:42.281895 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee95bae-9e7d-4cb2-978d-c40bc42eee96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:42.282818 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee95bae-9e7d-4cb2-978d-c40bc42eee96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:42.283100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee95bae-9e7d-4cb2-978d-c40bc42eee96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:42.283375 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee95bae-9e7d-4cb2-978d-c40bc42eee96 admin admin] PATH_INFO: `/v3/users/46cadf8ec0ba43e290d34f8a5521a22f` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:42.284576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee95bae-9e7d-4cb2-978d-c40bc42eee96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46cadf8ec0ba43e290d34f8a5521a22f)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:42.287065 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee95bae-9e7d-4cb2-978d-c40bc42eee96 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:42.601139 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7ee95bae-9e7d-4cb2-978d-c40bc42eee96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46cadf8ec0ba43e290d34f8a5521a22f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:42.690848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7ee95bae-9e7d-4cb2-978d-c40bc42eee96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46cadf8ec0ba43e290d34f8a5521a22f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:42.699352 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7ee95bae-9e7d-4cb2-978d-c40bc42eee96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46cadf8ec0ba43e290d34f8a5521a22f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:42.700084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7ee95bae-9e7d-4cb2-978d-c40bc42eee96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46cadf8ec0ba43e290d34f8a5521a22f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:42.719706 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7ee95bae-9e7d-4cb2-978d-c40bc42eee96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46cadf8ec0ba43e290d34f8a5521a22f'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:42.726358 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1279/2551] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:39:42 2022] DELETE /identity/v3/users/46cadf8ec0ba43e290d34f8a5521a22f => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:43.183673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f9fe7-52ee-4242-9d5d-c59fac1be563 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:43.185380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-de5f9fe7-52ee-4242-9d5d-c59fac1be563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:43.199043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f9fe7-52ee-4242-9d5d-c59fac1be563 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:43.204207 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:43.263520 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef02ec2-e38b-4a06-a21e-bf6485f1f908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:43.266140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef02ec2-e38b-4a06-a21e-bf6485f1f908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:43.266140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef02ec2-e38b-4a06-a21e-bf6485f1f908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:43.266140 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef02ec2-e38b-4a06-a21e-bf6485f1f908 admin admin] PATH_INFO: `/v3/projects/2dba3c55e04f4a209edbb7a20a485428` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:43.266561 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef02ec2-e38b-4a06-a21e-bf6485f1f908 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dba3c55e04f4a209edbb7a20a485428)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:43.268343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef02ec2-e38b-4a06-a21e-bf6485f1f908 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:43.392463 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-2ef02ec2-e38b-4a06-a21e-bf6485f1f908 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dba3c55e04f4a209edbb7a20a485428', 'request_id': 'req-2ef02ec2-e38b-4a06-a21e-bf6485f1f908'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:43.508074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-2ef02ec2-e38b-4a06-a21e-bf6485f1f908 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dba3c55e04f4a209edbb7a20a485428', 'request_id': 'req-2ef02ec2-e38b-4a06-a21e-bf6485f1f908'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:43.516310 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-2ef02ec2-e38b-4a06-a21e-bf6485f1f908 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dba3c55e04f4a209edbb7a20a485428', 'request_id': 'req-2ef02ec2-e38b-4a06-a21e-bf6485f1f908'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:43.527505 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-2ef02ec2-e38b-4a06-a21e-bf6485f1f908 admin admin] The token cache is being invalidate because project 2dba3c55e04f4a209edbb7a20a485428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:39:43.529220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-2ef02ec2-e38b-4a06-a21e-bf6485f1f908 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:43.529220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-2ef02ec2-e38b-4a06-a21e-bf6485f1f908 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:43.530481 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1273/2552] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:39:43 2022] DELETE /identity/v3/projects/2dba3c55e04f4a209edbb7a20a485428 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:43.535382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd12bc6-f7ee-46a4-b552-19bccbd7d103 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:43.535853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd12bc6-f7ee-46a4-b552-19bccbd7d103 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:43.535953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd12bc6-f7ee-46a4-b552-19bccbd7d103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:43.554143 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:43.593256 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-6bd12bc6-f7ee-46a4-b552-19bccbd7d103 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:43.594529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-6bd12bc6-f7ee-46a4-b552-19bccbd7d103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:43.655795 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1280/2553] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:43.663871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fb677-e832-468b-9be6-e0e166597d32 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:43.664156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fb677-e832-468b-9be6-e0e166597d32 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:43.664429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fb677-e832-468b-9be6-e0e166597d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:43.683731 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-2a9fb677-e832-468b-9be6-e0e166597d32 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:43.684942 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-2a9fb677-e832-468b-9be6-e0e166597d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:43.692904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1274/2554] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:43.700955 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd12bc6-f7ee-46a4-b552-19bccbd7d103 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:43.710971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd12bc6-f7ee-46a4-b552-19bccbd7d103 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:43.715623 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1924f-020b-4a8b-9d4e-85ce81621a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:43.716286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1924f-020b-4a8b-9d4e-85ce81621a92 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:43.716568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1924f-020b-4a8b-9d4e-85ce81621a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:43.716845 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1924f-020b-4a8b-9d4e-85ce81621a92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:43.717573 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1924f-020b-4a8b-9d4e-85ce81621a92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:39:43.717880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1924f-020b-4a8b-9d4e-85ce81621a92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:43.719522 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1924f-020b-4a8b-9d4e-85ce81621a92 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:43.731700 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1281/2555] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:39:43.736410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fb677-e832-468b-9be6-e0e166597d32 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:43.746956 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fb677-e832-468b-9be6-e0e166597d32 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:43.751295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39b4e8-531b-43a7-adb5-8681b8f199b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:43.751987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39b4e8-531b-43a7-adb5-8681b8f199b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:43.752267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39b4e8-531b-43a7-adb5-8681b8f199b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:43.752539 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39b4e8-531b-43a7-adb5-8681b8f199b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:43.753229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39b4e8-531b-43a7-adb5-8681b8f199b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:43.755148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39b4e8-531b-43a7-adb5-8681b8f199b3 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:43.930690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1275/2556] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:39:43 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:39:43.935785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1924f-020b-4a8b-9d4e-85ce81621a92 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:43.947166 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1924f-020b-4a8b-9d4e-85ce81621a92 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:44.006206 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d62c5c-4486-410a-b7d0-28eefccef235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:44.006946 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d62c5c-4486-410a-b7d0-28eefccef235 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:44.007227 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d62c5c-4486-410a-b7d0-28eefccef235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:44.007532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d62c5c-4486-410a-b7d0-28eefccef235 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:44.008233 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d62c5c-4486-410a-b7d0-28eefccef235 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:44.010228 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d62c5c-4486-410a-b7d0-28eefccef235 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:44.199683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1282/2557] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:39:43 2022] POST /identity/v3/users => generated 290 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:44.203771 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39b4e8-531b-43a7-adb5-8681b8f199b3 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:44.214939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39b4e8-531b-43a7-adb5-8681b8f199b3 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:44.219470 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6c132-41c0-4df0-a3bb-41653a0658f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:44.220119 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6c132-41c0-4df0-a3bb-41653a0658f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:44.225173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6c132-41c0-4df0-a3bb-41653a0658f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:44.225173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6c132-41c0-4df0-a3bb-41653a0658f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:44.225173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6c132-41c0-4df0-a3bb-41653a0658f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:44.225173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6c132-41c0-4df0-a3bb-41653a0658f8 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:44.233978 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1276/2558] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:44.238074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d62c5c-4486-410a-b7d0-28eefccef235 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:44.248313 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d62c5c-4486-410a-b7d0-28eefccef235 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:44.252839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404ce38-3287-4226-83d8-76302e2dcf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:44.253496 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404ce38-3287-4226-83d8-76302e2dcf4c admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:44.253779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404ce38-3287-4226-83d8-76302e2dcf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:44.254055 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404ce38-3287-4226-83d8-76302e2dcf4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:44.254818 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404ce38-3287-4226-83d8-76302e2dcf4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:44.256406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404ce38-3287-4226-83d8-76302e2dcf4c admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:44.266184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1283/2559] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:44.270222 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6c132-41c0-4df0-a3bb-41653a0658f8 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:44.280779 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6c132-41c0-4df0-a3bb-41653a0658f8 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:44.286559 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79b4c6-c45c-4d4e-8aea-770a217829e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:44.287580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79b4c6-c45c-4d4e-8aea-770a217829e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:44.287958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79b4c6-c45c-4d4e-8aea-770a217829e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:44.292642 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79b4c6-c45c-4d4e-8aea-770a217829e8 admin admin] PATH_INFO: `/v3/projects/8816eec119cc4fb3b16a92f74011fdbc/users/a3503ca2c3114293bdff9b36b23c2038/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:44.309290 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79b4c6-c45c-4d4e-8aea-770a217829e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8816eec119cc4fb3b16a92f74011fdbc, user_id=a3503ca2c3114293bdff9b36b23c2038, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:44.311985 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79b4c6-c45c-4d4e-8aea-770a217829e8 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:44.399885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1277/2560] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:44 2022] PUT /identity/v3/projects/8816eec119cc4fb3b16a92f74011fdbc/users/a3503ca2c3114293bdff9b36b23c2038/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:44.411953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182004c-7c36-41f8-b2dc-1962a7ae9bb1 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:44.426092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182004c-7c36-41f8-b2dc-1962a7ae9bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:44.426092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182004c-7c36-41f8-b2dc-1962a7ae9bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:44.441305 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-5182004c-7c36-41f8-b2dc-1962a7ae9bb1 None None] MFA Rules not processed for user `a3503ca2c3114293bdff9b36b23c2038`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:44.442727 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-5182004c-7c36-41f8-b2dc-1962a7ae9bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:44.489261 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:44.510139 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1284/2561] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:44.518454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79b4c6-c45c-4d4e-8aea-770a217829e8 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:44.558000 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:44.581454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79b4c6-c45c-4d4e-8aea-770a217829e8 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:44.586472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c860898-5050-4307-82a8-4a541d3bea11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:44.587133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c860898-5050-4307-82a8-4a541d3bea11 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:44.587419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c860898-5050-4307-82a8-4a541d3bea11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:44.587695 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c860898-5050-4307-82a8-4a541d3bea11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:44.588405 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c860898-5050-4307-82a8-4a541d3bea11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:44.590992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c860898-5050-4307-82a8-4a541d3bea11 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:44.775235 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1278/2562] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:39:44 2022] POST /identity/v3/projects => generated 379 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:44.777286 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182004c-7c36-41f8-b2dc-1962a7ae9bb1 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:44.789493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182004c-7c36-41f8-b2dc-1962a7ae9bb1 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:44.847345 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eed811-83fa-43d6-b562-f0d7a94128c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:44.848031 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eed811-83fa-43d6-b562-f0d7a94128c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:44.848311 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eed811-83fa-43d6-b562-f0d7a94128c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:44.848586 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eed811-83fa-43d6-b562-f0d7a94128c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:44.849293 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eed811-83fa-43d6-b562-f0d7a94128c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:44.851487 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eed811-83fa-43d6-b562-f0d7a94128c4 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:45.102284 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1285/2563] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:39:44 2022] POST /identity/v3/users => generated 290 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:45.106193 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c860898-5050-4307-82a8-4a541d3bea11 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:45.116410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c860898-5050-4307-82a8-4a541d3bea11 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:45.121260 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb2d18-59a8-4c1d-85c2-c87fa7ece517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:45.121997 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb2d18-59a8-4c1d-85c2-c87fa7ece517 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:45.122393 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb2d18-59a8-4c1d-85c2-c87fa7ece517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:45.122671 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb2d18-59a8-4c1d-85c2-c87fa7ece517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:45.123414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb2d18-59a8-4c1d-85c2-c87fa7ece517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:45.125007 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb2d18-59a8-4c1d-85c2-c87fa7ece517 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:45.136452 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1279/2564] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:45.140697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eed811-83fa-43d6-b562-f0d7a94128c4 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:45.151090 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eed811-83fa-43d6-b562-f0d7a94128c4 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:45.156612 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11cd2c-6004-45cb-8ca5-b628adfc2b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:45.157283 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11cd2c-6004-45cb-8ca5-b628adfc2b33 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:45.157570 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11cd2c-6004-45cb-8ca5-b628adfc2b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:45.157848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11cd2c-6004-45cb-8ca5-b628adfc2b33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:45.158608 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11cd2c-6004-45cb-8ca5-b628adfc2b33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:45.160213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11cd2c-6004-45cb-8ca5-b628adfc2b33 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:45.171197 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1286/2565] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:45.175477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb2d18-59a8-4c1d-85c2-c87fa7ece517 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:45.185969 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb2d18-59a8-4c1d-85c2-c87fa7ece517 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:45.191547 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e729693-7191-4d67-bd69-de733cde8153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:45.192620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e729693-7191-4d67-bd69-de733cde8153 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:45.192904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e729693-7191-4d67-bd69-de733cde8153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:45.193182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e729693-7191-4d67-bd69-de733cde8153 admin admin] PATH_INFO: `/v3/projects/fea827b9fa7f4a9ab9b37aef39ce5ae7/users/897f3fb4402c4931ba8fe092350ee96a/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:45.211316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e729693-7191-4d67-bd69-de733cde8153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fea827b9fa7f4a9ab9b37aef39ce5ae7, user_id=897f3fb4402c4931ba8fe092350ee96a, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:45.214938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e729693-7191-4d67-bd69-de733cde8153 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:45.444466 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1280/2566] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:45 2022] PUT /identity/v3/projects/fea827b9fa7f4a9ab9b37aef39ce5ae7/users/897f3fb4402c4931ba8fe092350ee96a/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:39:45.455543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1ae59-9677-4510-9bf2-f8ef2e1fc40b None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:45.455831 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1ae59-9677-4510-9bf2-f8ef2e1fc40b None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:45.456109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1ae59-9677-4510-9bf2-f8ef2e1fc40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:45.487714 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-32e1ae59-9677-4510-9bf2-f8ef2e1fc40b None None] MFA Rules not processed for user `897f3fb4402c4931ba8fe092350ee96a`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:45.489359 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-32e1ae59-9677-4510-9bf2-f8ef2e1fc40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:45.578950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1287/2567] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:45.583461 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e729693-7191-4d67-bd69-de733cde8153 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:45.653795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e729693-7191-4d67-bd69-de733cde8153 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:45.658853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e36e6-ca22-4b62-9eb0-cbb6c13f3a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:45.659616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e36e6-ca22-4b62-9eb0-cbb6c13f3a46 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:45.659940 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e36e6-ca22-4b62-9eb0-cbb6c13f3a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:45.660268 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e36e6-ca22-4b62-9eb0-cbb6c13f3a46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:45.661067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e36e6-ca22-4b62-9eb0-cbb6c13f3a46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:45.663167 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e36e6-ca22-4b62-9eb0-cbb6c13f3a46 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:45.937480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1281/2568] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:39:45 2022] POST /identity/v3/projects => generated 377 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:39:45.937600 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1ae59-9677-4510-9bf2-f8ef2e1fc40b None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:45.951893 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1ae59-9677-4510-9bf2-f8ef2e1fc40b None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:46.004315 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:46.013458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5935fec-3059-4dda-8108-f5a9d7738284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:46.014169 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5935fec-3059-4dda-8108-f5a9d7738284 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:46.014512 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5935fec-3059-4dda-8108-f5a9d7738284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:46.014789 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5935fec-3059-4dda-8108-f5a9d7738284 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:46.015494 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5935fec-3059-4dda-8108-f5a9d7738284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:46.017441 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5935fec-3059-4dda-8108-f5a9d7738284 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:46.188937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1288/2569] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:39:45 2022] POST /identity/v3/users => generated 295 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:39:46.193645 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e36e6-ca22-4b62-9eb0-cbb6c13f3a46 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:46.207191 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e36e6-ca22-4b62-9eb0-cbb6c13f3a46 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:46.211987 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf674498-261a-4c96-a34a-5b17b1ef0fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:46.212681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf674498-261a-4c96-a34a-5b17b1ef0fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:46.212933 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf674498-261a-4c96-a34a-5b17b1ef0fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:46.213280 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf674498-261a-4c96-a34a-5b17b1ef0fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:46.214470 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf674498-261a-4c96-a34a-5b17b1ef0fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:46.216069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf674498-261a-4c96-a34a-5b17b1ef0fa2 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:46.227350 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1282/2570] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:46.234185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5935fec-3059-4dda-8108-f5a9d7738284 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:46.245117 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5935fec-3059-4dda-8108-f5a9d7738284 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:46.253169 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384368a-6bd9-45d4-80ab-08fcce4ffe19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:46.254071 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384368a-6bd9-45d4-80ab-08fcce4ffe19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:46.254396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384368a-6bd9-45d4-80ab-08fcce4ffe19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:46.254676 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384368a-6bd9-45d4-80ab-08fcce4ffe19 admin admin] PATH_INFO: `/v3/projects/f98a8cf2343444028c5e8ee02c8ab040/users/d4656130b23b41598a9352a1da76c864/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:46.270966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384368a-6bd9-45d4-80ab-08fcce4ffe19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f98a8cf2343444028c5e8ee02c8ab040, user_id=d4656130b23b41598a9352a1da76c864, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:46.274521 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384368a-6bd9-45d4-80ab-08fcce4ffe19 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:46.429115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1289/2571] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:46 2022] PUT /identity/v3/projects/f98a8cf2343444028c5e8ee02c8ab040/users/d4656130b23b41598a9352a1da76c864/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:46.433176 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf674498-261a-4c96-a34a-5b17b1ef0fa2 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:46.443420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf674498-261a-4c96-a34a-5b17b1ef0fa2 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:46.501409 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ceaea1-2075-4731-a192-151a40efdd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:46.502059 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ceaea1-2075-4731-a192-151a40efdd36 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:46.502375 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ceaea1-2075-4731-a192-151a40efdd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:46.502656 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ceaea1-2075-4731-a192-151a40efdd36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:46.503385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ceaea1-2075-4731-a192-151a40efdd36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:46.504975 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ceaea1-2075-4731-a192-151a40efdd36 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:46.506320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:46.517305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1283/2572] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:46.521645 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384368a-6bd9-45d4-80ab-08fcce4ffe19 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:46.533626 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384368a-6bd9-45d4-80ab-08fcce4ffe19 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:46.538523 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51485c-c57e-40b3-b4f6-ee8fa7a7b966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:46.539458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51485c-c57e-40b3-b4f6-ee8fa7a7b966 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:46.539739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51485c-c57e-40b3-b4f6-ee8fa7a7b966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:46.540017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51485c-c57e-40b3-b4f6-ee8fa7a7b966 admin admin] PATH_INFO: `/v3/projects/f98a8cf2343444028c5e8ee02c8ab040/users/d4656130b23b41598a9352a1da76c864/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:46.542149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51485c-c57e-40b3-b4f6-ee8fa7a7b966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f98a8cf2343444028c5e8ee02c8ab040, user_id=d4656130b23b41598a9352a1da76c864, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:46.544879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51485c-c57e-40b3-b4f6-ee8fa7a7b966 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:46.557263 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1290/2573] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:46 2022] PUT /identity/v3/projects/f98a8cf2343444028c5e8ee02c8ab040/users/d4656130b23b41598a9352a1da76c864/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:46.567635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8ce7e-c1ec-43d4-9145-c1836d9167e5 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:46.567922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8ce7e-c1ec-43d4-9145-c1836d9167e5 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:46.568201 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8ce7e-c1ec-43d4-9145-c1836d9167e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:46.597916 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-f1f8ce7e-c1ec-43d4-9145-c1836d9167e5 None None] MFA Rules not processed for user `d4656130b23b41598a9352a1da76c864`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:46.599189 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f1f8ce7e-c1ec-43d4-9145-c1836d9167e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:46.673132 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1284/2574] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:46.684719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51485c-c57e-40b3-b4f6-ee8fa7a7b966 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:46.686457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ba51485c-c57e-40b3-b4f6-ee8fa7a7b966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:46.755932 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51485c-c57e-40b3-b4f6-ee8fa7a7b966 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:46.759746 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda0727-2bbf-41af-ad7f-c7763e212639 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:46.760918 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda0727-2bbf-41af-ad7f-c7763e212639 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:46.761227 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda0727-2bbf-41af-ad7f-c7763e212639 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:46.761502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda0727-2bbf-41af-ad7f-c7763e212639 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:46.772176 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda0727-2bbf-41af-ad7f-c7763e212639 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:46.773630 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda0727-2bbf-41af-ad7f-c7763e212639 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:46.784887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1291/2575] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:46.958844 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8ce7e-c1ec-43d4-9145-c1836d9167e5 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:46.958844 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f1f8ce7e-c1ec-43d4-9145-c1836d9167e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:46.962069 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8ce7e-c1ec-43d4-9145-c1836d9167e5 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:46.965772 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861cffe-33d2-474a-958f-4e9d3989c295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:46.966697 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861cffe-33d2-474a-958f-4e9d3989c295 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:46.966978 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861cffe-33d2-474a-958f-4e9d3989c295 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:46.967253 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861cffe-33d2-474a-958f-4e9d3989c295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:47.023081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a861cffe-33d2-474a-958f-4e9d3989c295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:47.024492 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a861cffe-33d2-474a-958f-4e9d3989c295 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:47.033341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1285/2576] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:53.270924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda0727-2bbf-41af-ad7f-c7763e212639 service cinder] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:53.342235 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda0727-2bbf-41af-ad7f-c7763e212639 service cinder] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:53.346954 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27e9e6-ecb6-4b1b-a6d3-a0fe5ac66602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:53.358666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27e9e6-ecb6-4b1b-a6d3-a0fe5ac66602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:53.358666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27e9e6-ecb6-4b1b-a6d3-a0fe5ac66602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:53.358666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27e9e6-ecb6-4b1b-a6d3-a0fe5ac66602 admin admin] PATH_INFO: `/v3/users/a3503ca2c3114293bdff9b36b23c2038` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:53.358666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27e9e6-ecb6-4b1b-a6d3-a0fe5ac66602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3503ca2c3114293bdff9b36b23c2038)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:53.358666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27e9e6-ecb6-4b1b-a6d3-a0fe5ac66602 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:53.630994 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7b27e9e6-ecb6-4b1b-a6d3-a0fe5ac66602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3503ca2c3114293bdff9b36b23c2038'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:53.633488 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:53.781492 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7b27e9e6-ecb6-4b1b-a6d3-a0fe5ac66602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3503ca2c3114293bdff9b36b23c2038'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:53.788962 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7b27e9e6-ecb6-4b1b-a6d3-a0fe5ac66602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3503ca2c3114293bdff9b36b23c2038'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:53.789607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7b27e9e6-ecb6-4b1b-a6d3-a0fe5ac66602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3503ca2c3114293bdff9b36b23c2038'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:53.802022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7b27e9e6-ecb6-4b1b-a6d3-a0fe5ac66602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3503ca2c3114293bdff9b36b23c2038'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:53.807928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1292/2577] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:39:53 2022] DELETE /identity/v3/users/a3503ca2c3114293bdff9b36b23c2038 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:53.812541 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ba492-9d3e-4410-869f-243cae5d50fa None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:53.813049 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ba492-9d3e-4410-869f-243cae5d50fa None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:53.813156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ba492-9d3e-4410-869f-243cae5d50fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:53.830812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-b34ba492-9d3e-4410-869f-243cae5d50fa None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:53.835239 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b34ba492-9d3e-4410-869f-243cae5d50fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:53.894975 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1286/2578] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:53.905477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27e9e6-ecb6-4b1b-a6d3-a0fe5ac66602 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:53.906969 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-7b27e9e6-ecb6-4b1b-a6d3-a0fe5ac66602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:53.985381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27e9e6-ecb6-4b1b-a6d3-a0fe5ac66602 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:53.988947 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21436d8c-abc6-43f7-9ee5-481161da57ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:53.989591 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21436d8c-abc6-43f7-9ee5-481161da57ca service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:53.989867 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21436d8c-abc6-43f7-9ee5-481161da57ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:53.990141 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21436d8c-abc6-43f7-9ee5-481161da57ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:53.999106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21436d8c-abc6-43f7-9ee5-481161da57ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:54.000522 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21436d8c-abc6-43f7-9ee5-481161da57ca service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:54.010148 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1293/2579] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:39:54.475701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ba492-9d3e-4410-869f-243cae5d50fa None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:54.477228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b34ba492-9d3e-4410-869f-243cae5d50fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:54.492998 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ba492-9d3e-4410-869f-243cae5d50fa None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:54.498759 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f4af6-a39b-4d6f-8f8d-444a6161e69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:54.499612 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f4af6-a39b-4d6f-8f8d-444a6161e69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:54.499898 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f4af6-a39b-4d6f-8f8d-444a6161e69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:54.500247 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f4af6-a39b-4d6f-8f8d-444a6161e69d admin admin] PATH_INFO: `/v3/projects/8816eec119cc4fb3b16a92f74011fdbc` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:54.501344 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f4af6-a39b-4d6f-8f8d-444a6161e69d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8816eec119cc4fb3b16a92f74011fdbc)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:54.503287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f4af6-a39b-4d6f-8f8d-444a6161e69d admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:54.520863 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:54.655969 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-397f4af6-a39b-4d6f-8f8d-444a6161e69d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8816eec119cc4fb3b16a92f74011fdbc', 'request_id': 'req-397f4af6-a39b-4d6f-8f8d-444a6161e69d'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:54.815670 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-397f4af6-a39b-4d6f-8f8d-444a6161e69d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8816eec119cc4fb3b16a92f74011fdbc', 'request_id': 'req-397f4af6-a39b-4d6f-8f8d-444a6161e69d'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:54.822675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-397f4af6-a39b-4d6f-8f8d-444a6161e69d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8816eec119cc4fb3b16a92f74011fdbc', 'request_id': 'req-397f4af6-a39b-4d6f-8f8d-444a6161e69d'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:54.834830 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-397f4af6-a39b-4d6f-8f8d-444a6161e69d admin admin] The token cache is being invalidate because project 8816eec119cc4fb3b16a92f74011fdbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:39:54.835124 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-397f4af6-a39b-4d6f-8f8d-444a6161e69d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:54.835789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-397f4af6-a39b-4d6f-8f8d-444a6161e69d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:54.837407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1287/2580] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:39:54 2022] DELETE /identity/v3/projects/8816eec119cc4fb3b16a92f74011fdbc => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:39:54.841474 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21436d8c-abc6-43f7-9ee5-481161da57ca service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:54.851106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21436d8c-abc6-43f7-9ee5-481161da57ca service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:54.915664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871c8c5-77af-4c2c-b615-c74bd7e4e412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:54.917070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871c8c5-77af-4c2c-b615-c74bd7e4e412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:54.917456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871c8c5-77af-4c2c-b615-c74bd7e4e412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:54.917850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871c8c5-77af-4c2c-b615-c74bd7e4e412 admin admin] PATH_INFO: `/v3/users/897f3fb4402c4931ba8fe092350ee96a` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:54.919090 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871c8c5-77af-4c2c-b615-c74bd7e4e412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=897f3fb4402c4931ba8fe092350ee96a)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:54.921007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871c8c5-77af-4c2c-b615-c74bd7e4e412 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:55.240467 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-8871c8c5-77af-4c2c-b615-c74bd7e4e412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '897f3fb4402c4931ba8fe092350ee96a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:55.244144 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:55.358717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-8871c8c5-77af-4c2c-b615-c74bd7e4e412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '897f3fb4402c4931ba8fe092350ee96a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:55.366084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-8871c8c5-77af-4c2c-b615-c74bd7e4e412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '897f3fb4402c4931ba8fe092350ee96a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:55.373631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-8871c8c5-77af-4c2c-b615-c74bd7e4e412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '897f3fb4402c4931ba8fe092350ee96a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:55.380406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-8871c8c5-77af-4c2c-b615-c74bd7e4e412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '897f3fb4402c4931ba8fe092350ee96a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:55.386562 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1294/2581] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:39:54 2022] DELETE /identity/v3/users/897f3fb4402c4931ba8fe092350ee96a => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:55.902056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f4af6-a39b-4d6f-8f8d-444a6161e69d admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:55.903597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-397f4af6-a39b-4d6f-8f8d-444a6161e69d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:55.922597 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f4af6-a39b-4d6f-8f8d-444a6161e69d admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:55.982443 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3ab2f-e077-42d9-84df-9960845a9f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:55.983305 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3ab2f-e077-42d9-84df-9960845a9f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:55.983587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3ab2f-e077-42d9-84df-9960845a9f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:55.983860 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3ab2f-e077-42d9-84df-9960845a9f10 admin admin] PATH_INFO: `/v3/projects/fea827b9fa7f4a9ab9b37aef39ce5ae7` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:55.985066 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3ab2f-e077-42d9-84df-9960845a9f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fea827b9fa7f4a9ab9b37aef39ce5ae7)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:55.987042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3ab2f-e077-42d9-84df-9960845a9f10 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:56.124584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-edb3ab2f-e077-42d9-84df-9960845a9f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fea827b9fa7f4a9ab9b37aef39ce5ae7', 'request_id': 'req-edb3ab2f-e077-42d9-84df-9960845a9f10'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:56.290148 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-edb3ab2f-e077-42d9-84df-9960845a9f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fea827b9fa7f4a9ab9b37aef39ce5ae7', 'request_id': 'req-edb3ab2f-e077-42d9-84df-9960845a9f10'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:56.297620 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-edb3ab2f-e077-42d9-84df-9960845a9f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fea827b9fa7f4a9ab9b37aef39ce5ae7', 'request_id': 'req-edb3ab2f-e077-42d9-84df-9960845a9f10'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:56.308324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-edb3ab2f-e077-42d9-84df-9960845a9f10 admin admin] The token cache is being invalidate because project fea827b9fa7f4a9ab9b37aef39ce5ae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:39:56.308615 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-edb3ab2f-e077-42d9-84df-9960845a9f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:56.309262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-edb3ab2f-e077-42d9-84df-9960845a9f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:56.310854 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1288/2582] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:39:55 2022] DELETE /identity/v3/projects/fea827b9fa7f4a9ab9b37aef39ce5ae7 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:56.314896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871c8c5-77af-4c2c-b615-c74bd7e4e412 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:56.316385 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8871c8c5-77af-4c2c-b615-c74bd7e4e412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:56.329728 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871c8c5-77af-4c2c-b615-c74bd7e4e412 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:56.389722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c52e30-0ccb-4ad6-8f88-8da058e791cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:56.390625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c52e30-0ccb-4ad6-8f88-8da058e791cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:56.390905 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c52e30-0ccb-4ad6-8f88-8da058e791cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:56.391181 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c52e30-0ccb-4ad6-8f88-8da058e791cf admin admin] PATH_INFO: `/v3/users/d4656130b23b41598a9352a1da76c864` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:56.392392 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c52e30-0ccb-4ad6-8f88-8da058e791cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4656130b23b41598a9352a1da76c864)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:56.394507 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c52e30-0ccb-4ad6-8f88-8da058e791cf admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:56.715806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-96c52e30-0ccb-4ad6-8f88-8da058e791cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4656130b23b41598a9352a1da76c864'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:56.861509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-96c52e30-0ccb-4ad6-8f88-8da058e791cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4656130b23b41598a9352a1da76c864'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:56.869138 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-96c52e30-0ccb-4ad6-8f88-8da058e791cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4656130b23b41598a9352a1da76c864'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:56.870285 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-96c52e30-0ccb-4ad6-8f88-8da058e791cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4656130b23b41598a9352a1da76c864'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:56.885301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-96c52e30-0ccb-4ad6-8f88-8da058e791cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4656130b23b41598a9352a1da76c864'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:56.892457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1295/2583] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:39:56 2022] DELETE /identity/v3/users/d4656130b23b41598a9352a1da76c864 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:57.464147 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3ab2f-e077-42d9-84df-9960845a9f10 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:57.465639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-edb3ab2f-e077-42d9-84df-9960845a9f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:57.480161 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3ab2f-e077-42d9-84df-9960845a9f10 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:57.511942 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:57.537295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d922b4-b109-4f9d-88bc-989d5259e909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:57.538152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d922b4-b109-4f9d-88bc-989d5259e909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:57.538463 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d922b4-b109-4f9d-88bc-989d5259e909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:57.538746 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d922b4-b109-4f9d-88bc-989d5259e909 admin admin] PATH_INFO: `/v3/projects/f98a8cf2343444028c5e8ee02c8ab040` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:57.539936 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d922b4-b109-4f9d-88bc-989d5259e909 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f98a8cf2343444028c5e8ee02c8ab040)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:57.542476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d922b4-b109-4f9d-88bc-989d5259e909 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:57.724449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-41d922b4-b109-4f9d-88bc-989d5259e909 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f98a8cf2343444028c5e8ee02c8ab040', 'request_id': 'req-41d922b4-b109-4f9d-88bc-989d5259e909'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:57.941777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-41d922b4-b109-4f9d-88bc-989d5259e909 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f98a8cf2343444028c5e8ee02c8ab040', 'request_id': 'req-41d922b4-b109-4f9d-88bc-989d5259e909'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:57.949428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-41d922b4-b109-4f9d-88bc-989d5259e909 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f98a8cf2343444028c5e8ee02c8ab040', 'request_id': 'req-41d922b4-b109-4f9d-88bc-989d5259e909'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:57.960082 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-41d922b4-b109-4f9d-88bc-989d5259e909 admin admin] The token cache is being invalidate because project f98a8cf2343444028c5e8ee02c8ab040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:39:57.963157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-41d922b4-b109-4f9d-88bc-989d5259e909 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:57.963157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-41d922b4-b109-4f9d-88bc-989d5259e909 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:39:57.963157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1289/2584] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:39:57 2022] DELETE /identity/v3/projects/f98a8cf2343444028c5e8ee02c8ab040 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:57.969374 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615b745-6bf6-4059-bc06-c23b6e3d42fe None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:57.969889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615b745-6bf6-4059-bc06-c23b6e3d42fe None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:57.970000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615b745-6bf6-4059-bc06-c23b6e3d42fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:57.989064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-d615b745-6bf6-4059-bc06-c23b6e3d42fe None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:57.990287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d615b745-6bf6-4059-bc06-c23b6e3d42fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:58.039732 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:58.054799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1296/2585] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:58.064216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd050f-0de7-4171-b374-afe14706807d None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:58.064570 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd050f-0de7-4171-b374-afe14706807d None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:58.066222 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd050f-0de7-4171-b374-afe14706807d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:58.083704 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-c1dd050f-0de7-4171-b374-afe14706807d None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:58.084916 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-c1dd050f-0de7-4171-b374-afe14706807d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:58.094323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1290/2586] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:58.098394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615b745-6bf6-4059-bc06-c23b6e3d42fe None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:58.108054 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615b745-6bf6-4059-bc06-c23b6e3d42fe None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:58.113598 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d14def-b216-4c52-a5c5-179bf2ea924c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:58.113598 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d14def-b216-4c52-a5c5-179bf2ea924c admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:58.113598 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d14def-b216-4c52-a5c5-179bf2ea924c admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:58.113922 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d14def-b216-4c52-a5c5-179bf2ea924c admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:58.114665 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d14def-b216-4c52-a5c5-179bf2ea924c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:39:58.114975 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d14def-b216-4c52-a5c5-179bf2ea924c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:58.116570 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d14def-b216-4c52-a5c5-179bf2ea924c admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:58.128035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1297/2587] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:39:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:39:58.132531 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd050f-0de7-4171-b374-afe14706807d None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:58.142464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd050f-0de7-4171-b374-afe14706807d None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:58.147052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160bc254-8595-44bf-9483-b1405eba71ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:58.151502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bc254-8595-44bf-9483-b1405eba71ec admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:58.151502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bc254-8595-44bf-9483-b1405eba71ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:58.151502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bc254-8595-44bf-9483-b1405eba71ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:58.151502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160bc254-8595-44bf-9483-b1405eba71ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:58.151502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160bc254-8595-44bf-9483-b1405eba71ec admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:58.390110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1291/2588] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:39:58 2022] POST /identity/v3/projects => generated 381 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:39:58.394696 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d14def-b216-4c52-a5c5-179bf2ea924c admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:58.405301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d14def-b216-4c52-a5c5-179bf2ea924c admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:58.461423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09317e-2d79-4785-a6d3-abf604047b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:58.462085 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09317e-2d79-4785-a6d3-abf604047b3c admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:58.462395 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09317e-2d79-4785-a6d3-abf604047b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:58.462673 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09317e-2d79-4785-a6d3-abf604047b3c admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:58.463342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09317e-2d79-4785-a6d3-abf604047b3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:58.465397 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09317e-2d79-4785-a6d3-abf604047b3c admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:58.622430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1298/2589] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:39:58 2022] POST /identity/v3/users => generated 291 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:39:58.626549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160bc254-8595-44bf-9483-b1405eba71ec admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:58.636616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160bc254-8595-44bf-9483-b1405eba71ec admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:58.643820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09c1cd-8576-44bb-ba45-0e1bbb88bd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:58.643820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09c1cd-8576-44bb-ba45-0e1bbb88bd87 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:58.643820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09c1cd-8576-44bb-ba45-0e1bbb88bd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:58.643820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09c1cd-8576-44bb-ba45-0e1bbb88bd87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:58.644323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e09c1cd-8576-44bb-ba45-0e1bbb88bd87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:58.645058 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e09c1cd-8576-44bb-ba45-0e1bbb88bd87 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:58.655837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1292/2590] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:58.660127 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09317e-2d79-4785-a6d3-abf604047b3c admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:58.671849 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09317e-2d79-4785-a6d3-abf604047b3c admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:58.676692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becf847-159d-4324-bd9a-4e7ee363b9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:58.677382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becf847-159d-4324-bd9a-4e7ee363b9fb admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:58.677669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becf847-159d-4324-bd9a-4e7ee363b9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:58.677946 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becf847-159d-4324-bd9a-4e7ee363b9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:58.678732 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7becf847-159d-4324-bd9a-4e7ee363b9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:58.680358 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7becf847-159d-4324-bd9a-4e7ee363b9fb admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:58.691302 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1299/2591] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:39:58.695512 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09c1cd-8576-44bb-ba45-0e1bbb88bd87 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:58.705771 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09c1cd-8576-44bb-ba45-0e1bbb88bd87 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:58.710464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c82ad6-a9ab-47a7-830d-e6845bae910d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:58.711355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c82ad6-a9ab-47a7-830d-e6845bae910d admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:58.711675 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c82ad6-a9ab-47a7-830d-e6845bae910d admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:58.711958 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c82ad6-a9ab-47a7-830d-e6845bae910d admin admin] PATH_INFO: `/v3/projects/644a224ca9e34a2780d0d15c47bd2ca8/users/86e03c4bfea1484ebd099fbeeccbe05c/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:58.727628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c82ad6-a9ab-47a7-830d-e6845bae910d admin admin] RBAC: Authorizing `identity:create_grant(project_id=644a224ca9e34a2780d0d15c47bd2ca8, user_id=86e03c4bfea1484ebd099fbeeccbe05c, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:58.730800 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c82ad6-a9ab-47a7-830d-e6845bae910d admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:58.907915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1293/2592] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:39:58 2022] PUT /identity/v3/projects/644a224ca9e34a2780d0d15c47bd2ca8/users/86e03c4bfea1484ebd099fbeeccbe05c/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:39:58.919444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9b7ae-eb28-4ceb-9a4c-b38bcd2e9d54 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:58.919758 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9b7ae-eb28-4ceb-9a4c-b38bcd2e9d54 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:58.920045 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9b7ae-eb28-4ceb-9a4c-b38bcd2e9d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:58.946767 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-43f9b7ae-eb28-4ceb-9a4c-b38bcd2e9d54 None None] MFA Rules not processed for user `86e03c4bfea1484ebd099fbeeccbe05c`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:39:58.947988 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-43f9b7ae-eb28-4ceb-9a4c-b38bcd2e9d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:59.011115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1300/2593] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:39:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:39:59.022984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c82ad6-a9ab-47a7-830d-e6845bae910d admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:39:59.024673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a3c82ad6-a9ab-47a7-830d-e6845bae910d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:39:59.088912 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:39:59.103476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c82ad6-a9ab-47a7-830d-e6845bae910d admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:39:59.107870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0bcdd-14d5-400f-9014-e8d4963e8043 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:39:59.108549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0bcdd-14d5-400f-9014-e8d4963e8043 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:39:59.108836 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0bcdd-14d5-400f-9014-e8d4963e8043 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:39:59.109254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0bcdd-14d5-400f-9014-e8d4963e8043 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:39:59.117835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0bcdd-14d5-400f-9014-e8d4963e8043 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:39:59.119301 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0bcdd-14d5-400f-9014-e8d4963e8043 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:39:59.127990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1294/2594] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:39:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:40:09.008296 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:09.260805 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=8 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:09.754066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9b7ae-eb28-4ceb-9a4c-b38bcd2e9d54 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:09.833212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9b7ae-eb28-4ceb-9a4c-b38bcd2e9d54 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:09.839160 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6232e4-2171-4598-9e51-b5a873dbf12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:09.840189 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6232e4-2171-4598-9e51-b5a873dbf12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:09.840475 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6232e4-2171-4598-9e51-b5a873dbf12c admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:09.840752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6232e4-2171-4598-9e51-b5a873dbf12c admin admin] PATH_INFO: `/v3/users/86e03c4bfea1484ebd099fbeeccbe05c` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:09.842048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6232e4-2171-4598-9e51-b5a873dbf12c admin admin] RBAC: Authorizing `identity:delete_user(user_id=86e03c4bfea1484ebd099fbeeccbe05c)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:09.844077 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6232e4-2171-4598-9e51-b5a873dbf12c admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:10.124093 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-8e6232e4-2171-4598-9e51-b5a873dbf12c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86e03c4bfea1484ebd099fbeeccbe05c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:10.244022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-8e6232e4-2171-4598-9e51-b5a873dbf12c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86e03c4bfea1484ebd099fbeeccbe05c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:10.252175 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-8e6232e4-2171-4598-9e51-b5a873dbf12c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86e03c4bfea1484ebd099fbeeccbe05c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:10.252854 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-8e6232e4-2171-4598-9e51-b5a873dbf12c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86e03c4bfea1484ebd099fbeeccbe05c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:10.266245 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-8e6232e4-2171-4598-9e51-b5a873dbf12c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86e03c4bfea1484ebd099fbeeccbe05c'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:10.272885 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1301/2595] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:40:09 2022] DELETE /identity/v3/users/86e03c4bfea1484ebd099fbeeccbe05c => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:10.277494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204710eb-6c93-4994-a795-6360b952e8e4 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:10.278007 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204710eb-6c93-4994-a795-6360b952e8e4 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:10.278107 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204710eb-6c93-4994-a795-6360b952e8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:10.298564 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-204710eb-6c93-4994-a795-6360b952e8e4 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:10.300816 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-204710eb-6c93-4994-a795-6360b952e8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:10.363344 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1295/2596] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:10.372715 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6232e4-2171-4598-9e51-b5a873dbf12c admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:10.374362 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8e6232e4-2171-4598-9e51-b5a873dbf12c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:10.436142 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6232e4-2171-4598-9e51-b5a873dbf12c admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:10.439808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b64ce5-6c3d-467a-bc2d-971cc9456ca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:10.440545 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64ce5-6c3d-467a-bc2d-971cc9456ca2 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:10.440890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64ce5-6c3d-467a-bc2d-971cc9456ca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:10.441178 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64ce5-6c3d-467a-bc2d-971cc9456ca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:10.451239 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b64ce5-6c3d-467a-bc2d-971cc9456ca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:10.452695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b64ce5-6c3d-467a-bc2d-971cc9456ca2 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:10.464790 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1302/2597] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:40:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:40:10.477381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204710eb-6c93-4994-a795-6360b952e8e4 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:10.478996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-204710eb-6c93-4994-a795-6360b952e8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:10.586038 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204710eb-6c93-4994-a795-6360b952e8e4 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:10.591263 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609f002-394f-41f2-867d-50bdcfa2a443 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2192547babde4449a46f9cb37ed64524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:10.591917 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609f002-394f-41f2-867d-50bdcfa2a443 service placement] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:10.592198 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609f002-394f-41f2-867d-50bdcfa2a443 service placement] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:10.592472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609f002-394f-41f2-867d-50bdcfa2a443 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:10.594284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e609f002-394f-41f2-867d-50bdcfa2a443 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:10.605934 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609f002-394f-41f2-867d-50bdcfa2a443 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:10.609487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609f002-394f-41f2-867d-50bdcfa2a443 service placement] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:10.620026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1296/2598] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:40:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:40:11.000436 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b64ce5-6c3d-467a-bc2d-971cc9456ca2 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:11.002074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-03b64ce5-6c3d-467a-bc2d-971cc9456ca2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:11.017363 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b64ce5-6c3d-467a-bc2d-971cc9456ca2 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:11.023269 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f097b-9e75-4959-a76d-3cd56c67b6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:11.024990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f097b-9e75-4959-a76d-3cd56c67b6be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:11.024990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f097b-9e75-4959-a76d-3cd56c67b6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:11.024990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f097b-9e75-4959-a76d-3cd56c67b6be admin admin] PATH_INFO: `/v3/projects/644a224ca9e34a2780d0d15c47bd2ca8` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:11.026042 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f097b-9e75-4959-a76d-3cd56c67b6be admin admin] RBAC: Authorizing `identity:delete_project(project_id=644a224ca9e34a2780d0d15c47bd2ca8)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:11.028209 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f097b-9e75-4959-a76d-3cd56c67b6be admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:11.047473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:11.166544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-2d7f097b-9e75-4959-a76d-3cd56c67b6be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '644a224ca9e34a2780d0d15c47bd2ca8', 'request_id': 'req-2d7f097b-9e75-4959-a76d-3cd56c67b6be'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:11.306500 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-2d7f097b-9e75-4959-a76d-3cd56c67b6be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '644a224ca9e34a2780d0d15c47bd2ca8', 'request_id': 'req-2d7f097b-9e75-4959-a76d-3cd56c67b6be'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:11.314120 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-2d7f097b-9e75-4959-a76d-3cd56c67b6be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '644a224ca9e34a2780d0d15c47bd2ca8', 'request_id': 'req-2d7f097b-9e75-4959-a76d-3cd56c67b6be'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:11.324733 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-2d7f097b-9e75-4959-a76d-3cd56c67b6be admin admin] The token cache is being invalidate because project 644a224ca9e34a2780d0d15c47bd2ca8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:40:11.325019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-2d7f097b-9e75-4959-a76d-3cd56c67b6be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:11.326233 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-2d7f097b-9e75-4959-a76d-3cd56c67b6be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:11.327911 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1303/2599] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:40:10 2022] DELETE /identity/v3/projects/644a224ca9e34a2780d0d15c47bd2ca8 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:40:15.698795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609f002-394f-41f2-867d-50bdcfa2a443 service placement] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:15.700226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e609f002-394f-41f2-867d-50bdcfa2a443 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:15.712349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609f002-394f-41f2-867d-50bdcfa2a443 service placement] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:15.761869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800d64b-5e7c-44a0-b756-8e0154693131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:15.762549 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800d64b-5e7c-44a0-b756-8e0154693131 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:15.762835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800d64b-5e7c-44a0-b756-8e0154693131 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:15.763108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800d64b-5e7c-44a0-b756-8e0154693131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:15.765511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-c800d64b-5e7c-44a0-b756-8e0154693131 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:15.790736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:15.829616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800d64b-5e7c-44a0-b756-8e0154693131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:15.831195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800d64b-5e7c-44a0-b756-8e0154693131 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:15.841483 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1297/2600] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:40:19.349905 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f097b-9e75-4959-a76d-3cd56c67b6be admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:19.351498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2d7f097b-9e75-4959-a76d-3cd56c67b6be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:19.424384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f097b-9e75-4959-a76d-3cd56c67b6be admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:19.429474 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71326d20-794f-4f96-b098-60476ecbbd61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2192547babde4449a46f9cb37ed64524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:19.430148 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71326d20-794f-4f96-b098-60476ecbbd61 service placement] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:19.430462 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71326d20-794f-4f96-b098-60476ecbbd61 service placement] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:19.430744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71326d20-794f-4f96-b098-60476ecbbd61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:19.432529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-71326d20-794f-4f96-b098-60476ecbbd61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:19.444788 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71326d20-794f-4f96-b098-60476ecbbd61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:19.446395 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71326d20-794f-4f96-b098-60476ecbbd61 service placement] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:19.457048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1304/2601] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:40:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:40:19.487711 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800d64b-5e7c-44a0-b756-8e0154693131 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:19.489686 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-c800d64b-5e7c-44a0-b756-8e0154693131 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:19.505342 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800d64b-5e7c-44a0-b756-8e0154693131 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:19.510757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7f783-2535-47c2-881d-eea0aab13622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:19.510976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7f783-2535-47c2-881d-eea0aab13622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:19.511241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7f783-2535-47c2-881d-eea0aab13622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:19.511518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7f783-2535-47c2-881d-eea0aab13622 admin admin] PATH_INFO: `/v3/users/6498ef5c517c4fa4abed90e430936ded` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:19.512736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7f783-2535-47c2-881d-eea0aab13622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6498ef5c517c4fa4abed90e430936ded)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:19.514778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7f783-2535-47c2-881d-eea0aab13622 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:19.702857 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-46b7f783-2535-47c2-881d-eea0aab13622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6498ef5c517c4fa4abed90e430936ded'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:19.751778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-46b7f783-2535-47c2-881d-eea0aab13622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6498ef5c517c4fa4abed90e430936ded'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:19.752414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-46b7f783-2535-47c2-881d-eea0aab13622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6498ef5c517c4fa4abed90e430936ded'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:19.765459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-46b7f783-2535-47c2-881d-eea0aab13622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6498ef5c517c4fa4abed90e430936ded'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:19.772548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-46b7f783-2535-47c2-881d-eea0aab13622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6498ef5c517c4fa4abed90e430936ded'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:19.778859 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1298/2602] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:40:19 2022] DELETE /identity/v3/users/6498ef5c517c4fa4abed90e430936ded => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:20.041365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71326d20-794f-4f96-b098-60476ecbbd61 service placement] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:20.051703 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71326d20-794f-4f96-b098-60476ecbbd61 service placement] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:20.114683 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3dd94f-aa88-4921-8742-61aac116408b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:20.115571 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3dd94f-aa88-4921-8742-61aac116408b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:20.115850 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3dd94f-aa88-4921-8742-61aac116408b admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:20.116124 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3dd94f-aa88-4921-8742-61aac116408b admin admin] PATH_INFO: `/v3/projects/4446d4a6f4e64ccb8bfd5b3fe6f2127a` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:20.117304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3dd94f-aa88-4921-8742-61aac116408b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4446d4a6f4e64ccb8bfd5b3fe6f2127a)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:20.119251 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3dd94f-aa88-4921-8742-61aac116408b admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:20.246781 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:20.254909 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-bc3dd94f-aa88-4921-8742-61aac116408b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4446d4a6f4e64ccb8bfd5b3fe6f2127a', 'request_id': 'req-bc3dd94f-aa88-4921-8742-61aac116408b'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:20.340567 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-bc3dd94f-aa88-4921-8742-61aac116408b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4446d4a6f4e64ccb8bfd5b3fe6f2127a', 'request_id': 'req-bc3dd94f-aa88-4921-8742-61aac116408b'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:20.347780 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-bc3dd94f-aa88-4921-8742-61aac116408b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4446d4a6f4e64ccb8bfd5b3fe6f2127a', 'request_id': 'req-bc3dd94f-aa88-4921-8742-61aac116408b'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:20.358603 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-bc3dd94f-aa88-4921-8742-61aac116408b admin admin] The token cache is being invalidate because project 4446d4a6f4e64ccb8bfd5b3fe6f2127a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:40:20.359196 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-bc3dd94f-aa88-4921-8742-61aac116408b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:20.359695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-bc3dd94f-aa88-4921-8742-61aac116408b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:20.361239 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1305/2603] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:40:20 2022] DELETE /identity/v3/projects/4446d4a6f4e64ccb8bfd5b3fe6f2127a => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:20.366142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2ca76-daed-4d6b-89f9-7f5625ffc911 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:20.366460 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2ca76-daed-4d6b-89f9-7f5625ffc911 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:20.366739 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2ca76-daed-4d6b-89f9-7f5625ffc911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:20.385022 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-09d2ca76-daed-4d6b-89f9-7f5625ffc911 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:20.386310 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-09d2ca76-daed-4d6b-89f9-7f5625ffc911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:20.444647 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1299/2604] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:40:20.452666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d1bc3-7bd9-4880-a8fc-2bda78eba4e7 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:20.452949 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d1bc3-7bd9-4880-a8fc-2bda78eba4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:20.453223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d1bc3-7bd9-4880-a8fc-2bda78eba4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:20.471987 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-a34d1bc3-7bd9-4880-a8fc-2bda78eba4e7 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:20.473222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-a34d1bc3-7bd9-4880-a8fc-2bda78eba4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:20.481275 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1306/2605] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:20.485429 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2ca76-daed-4d6b-89f9-7f5625ffc911 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:20.495193 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2ca76-daed-4d6b-89f9-7f5625ffc911 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:20.499745 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c1e67-18af-4f39-a07b-8a75d978f42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:20.500399 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c1e67-18af-4f39-a07b-8a75d978f42c admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:20.500679 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c1e67-18af-4f39-a07b-8a75d978f42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:20.501053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c1e67-18af-4f39-a07b-8a75d978f42c admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:20.501680 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c1e67-18af-4f39-a07b-8a75d978f42c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:40:20.501983 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c1e67-18af-4f39-a07b-8a75d978f42c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:20.503649 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c1e67-18af-4f39-a07b-8a75d978f42c admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:20.516232 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1300/2606] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:40:20.520616 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d1bc3-7bd9-4880-a8fc-2bda78eba4e7 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:20.530681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d1bc3-7bd9-4880-a8fc-2bda78eba4e7 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:20.535323 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c3b39-9090-43ac-8e2a-576eb6ea31fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:20.535976 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c3b39-9090-43ac-8e2a-576eb6ea31fa admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:20.536258 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c3b39-9090-43ac-8e2a-576eb6ea31fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:20.536531 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c3b39-9090-43ac-8e2a-576eb6ea31fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:20.537228 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c3b39-9090-43ac-8e2a-576eb6ea31fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:20.539221 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c3b39-9090-43ac-8e2a-576eb6ea31fa admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:20.690512 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1307/2607] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:40:20 2022] POST /identity/v3/projects => generated 405 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:40:20.694427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c1e67-18af-4f39-a07b-8a75d978f42c admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:20.705676 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c1e67-18af-4f39-a07b-8a75d978f42c admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:20.723435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:20.771551 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca401129-5a11-49f3-8531-5b6a8ca7c1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:20.772318 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca401129-5a11-49f3-8531-5b6a8ca7c1e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:20.772617 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca401129-5a11-49f3-8531-5b6a8ca7c1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:20.772902 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca401129-5a11-49f3-8531-5b6a8ca7c1e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:20.773627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca401129-5a11-49f3-8531-5b6a8ca7c1e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:20.775556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca401129-5a11-49f3-8531-5b6a8ca7c1e9 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:20.929456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1301/2608] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:40:20 2022] POST /identity/v3/users => generated 303 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:40:20.933748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c3b39-9090-43ac-8e2a-576eb6ea31fa admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:20.946929 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c3b39-9090-43ac-8e2a-576eb6ea31fa admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:20.951230 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f11d8-ffc1-4869-9104-76740effa540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:20.952495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f11d8-ffc1-4869-9104-76740effa540 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:20.952772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f11d8-ffc1-4869-9104-76740effa540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:20.953041 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f11d8-ffc1-4869-9104-76740effa540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:20.953767 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0f11d8-ffc1-4869-9104-76740effa540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:20.955664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0f11d8-ffc1-4869-9104-76740effa540 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:20.967288 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1308/2609] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:40:20.970992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca401129-5a11-49f3-8531-5b6a8ca7c1e9 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:20.981426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca401129-5a11-49f3-8531-5b6a8ca7c1e9 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:20.986682 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c32eda-8eb8-4734-a9af-3a40cdb477e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:20.987353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c32eda-8eb8-4734-a9af-3a40cdb477e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:20.987640 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c32eda-8eb8-4734-a9af-3a40cdb477e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:20.987922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c32eda-8eb8-4734-a9af-3a40cdb477e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:20.988653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c32eda-8eb8-4734-a9af-3a40cdb477e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:20.990236 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c32eda-8eb8-4734-a9af-3a40cdb477e0 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:21.001108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1302/2610] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:40:21.005754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f11d8-ffc1-4869-9104-76740effa540 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:21.016083 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f11d8-ffc1-4869-9104-76740effa540 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:21.020762 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c073f-d2c0-4154-af87-16b0dfc32663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:21.021646 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c073f-d2c0-4154-af87-16b0dfc32663 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:21.021929 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c073f-d2c0-4154-af87-16b0dfc32663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:21.022206 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c073f-d2c0-4154-af87-16b0dfc32663 admin admin] PATH_INFO: `/v3/projects/39ad4f66e6434dc98537140e19974111/users/8c97b76fc0394966a76eaad5b30bc496/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:21.036661 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c073f-d2c0-4154-af87-16b0dfc32663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ad4f66e6434dc98537140e19974111, user_id=8c97b76fc0394966a76eaad5b30bc496, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:21.039346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c073f-d2c0-4154-af87-16b0dfc32663 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:21.140726 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1309/2611] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:40:21 2022] PUT /identity/v3/projects/39ad4f66e6434dc98537140e19974111/users/8c97b76fc0394966a76eaad5b30bc496/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:21.145757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c99ad9-9d5f-4b39-a912-8810cccda255 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:21.146052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c99ad9-9d5f-4b39-a912-8810cccda255 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:21.146432 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c99ad9-9d5f-4b39-a912-8810cccda255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:21.164248 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-39c99ad9-9d5f-4b39-a912-8810cccda255 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:21.165458 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-39c99ad9-9d5f-4b39-a912-8810cccda255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:21.223010 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1303/2612] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:21.232456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c073f-d2c0-4154-af87-16b0dfc32663 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:21.234693 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-336c073f-d2c0-4154-af87-16b0dfc32663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:21.291399 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c073f-d2c0-4154-af87-16b0dfc32663 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:21.295185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fe11e-01f1-447c-b301-d833c36616f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:21.296365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fe11e-01f1-447c-b301-d833c36616f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:21.296650 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fe11e-01f1-447c-b301-d833c36616f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:21.296929 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fe11e-01f1-447c-b301-d833c36616f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:21.306032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fe11e-01f1-447c-b301-d833c36616f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:21.307520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fe11e-01f1-447c-b301-d833c36616f6 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:21.317959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1310/2613] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:40:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:40:23.062514 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b50b4-296f-4f9c-bda0-a8463d4dd23a None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:23.063121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b50b4-296f-4f9c-bda0-a8463d4dd23a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:23.063121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b50b4-296f-4f9c-bda0-a8463d4dd23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:23.097777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-dc0b50b4-296f-4f9c-bda0-a8463d4dd23a None None] MFA Rules not processed for user `8c97b76fc0394966a76eaad5b30bc496`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:23.099135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-dc0b50b4-296f-4f9c-bda0-a8463d4dd23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:23.167315 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1304/2614] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:23 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:23.171927 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fe11e-01f1-447c-b301-d833c36616f6 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:23.181925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fe11e-01f1-447c-b301-d833c36616f6 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:23.187460 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1bea2-6044-457b-9d7f-028833215e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:23.188119 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1bea2-6044-457b-9d7f-028833215e70 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:23.188401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1bea2-6044-457b-9d7f-028833215e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:23.188709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1bea2-6044-457b-9d7f-028833215e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:23.189414 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1bea2-6044-457b-9d7f-028833215e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:23.191360 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1bea2-6044-457b-9d7f-028833215e70 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:23.279687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1311/2615] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:40:23 2022] POST /identity/v3/projects => generated 403 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:40:23.283861 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b50b4-296f-4f9c-bda0-a8463d4dd23a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:23.295934 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b50b4-296f-4f9c-bda0-a8463d4dd23a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:23.321779 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:23.351525 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb43a6-8ea6-4483-84a4-f206930dd4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:23.352194 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb43a6-8ea6-4483-84a4-f206930dd4ed admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:23.352478 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb43a6-8ea6-4483-84a4-f206930dd4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:23.352750 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb43a6-8ea6-4483-84a4-f206930dd4ed admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:23.353444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb43a6-8ea6-4483-84a4-f206930dd4ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:23.355515 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb43a6-8ea6-4483-84a4-f206930dd4ed admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:23.467162 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1305/2616] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:40:23 2022] POST /identity/v3/users => generated 308 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:40:23.471214 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1bea2-6044-457b-9d7f-028833215e70 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:23.481512 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1bea2-6044-457b-9d7f-028833215e70 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:23.486780 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d82f56-8132-4000-b1c9-0859077e20a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:23.487507 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d82f56-8132-4000-b1c9-0859077e20a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:23.487794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d82f56-8132-4000-b1c9-0859077e20a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:23.488094 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d82f56-8132-4000-b1c9-0859077e20a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:23.488798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d82f56-8132-4000-b1c9-0859077e20a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:23.490430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d82f56-8132-4000-b1c9-0859077e20a1 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:23.501872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1312/2617] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:40:23.506143 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb43a6-8ea6-4483-84a4-f206930dd4ed admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:23.516345 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb43a6-8ea6-4483-84a4-f206930dd4ed admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:23.521050 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d0ea9-5243-473b-8479-cda86933d7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:23.522136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d0ea9-5243-473b-8479-cda86933d7f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:23.522375 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d0ea9-5243-473b-8479-cda86933d7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:23.522665 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d0ea9-5243-473b-8479-cda86933d7f0 admin admin] PATH_INFO: `/v3/projects/64528a378a7b4ed58d17564e043593d3/users/6a95fd43d54e4cd0986b5ea2394e6e69/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:23.538424 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d0ea9-5243-473b-8479-cda86933d7f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64528a378a7b4ed58d17564e043593d3, user_id=6a95fd43d54e4cd0986b5ea2394e6e69, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:23.541249 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d0ea9-5243-473b-8479-cda86933d7f0 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:23.638082 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1306/2618] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:40:23 2022] PUT /identity/v3/projects/64528a378a7b4ed58d17564e043593d3/users/6a95fd43d54e4cd0986b5ea2394e6e69/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:23.643405 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d82f56-8132-4000-b1c9-0859077e20a1 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:23.652306 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d82f56-8132-4000-b1c9-0859077e20a1 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:23.681817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:23.712031 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd86e60-ce53-44be-b38d-44a4f268c8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:23.712704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd86e60-ce53-44be-b38d-44a4f268c8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:23.712986 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd86e60-ce53-44be-b38d-44a4f268c8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:23.713261 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd86e60-ce53-44be-b38d-44a4f268c8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:23.713978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd86e60-ce53-44be-b38d-44a4f268c8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:23.715602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd86e60-ce53-44be-b38d-44a4f268c8e7 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:23.726474 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1313/2619] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:40:23.730796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d0ea9-5243-473b-8479-cda86933d7f0 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:23.741867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d0ea9-5243-473b-8479-cda86933d7f0 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:23.746636 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5845d-2c19-404c-8e14-63bb0aadaedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:23.747517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5845d-2c19-404c-8e14-63bb0aadaedc admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:23.747803 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5845d-2c19-404c-8e14-63bb0aadaedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:23.748079 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5845d-2c19-404c-8e14-63bb0aadaedc admin admin] PATH_INFO: `/v3/projects/64528a378a7b4ed58d17564e043593d3/users/6a95fd43d54e4cd0986b5ea2394e6e69/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:23.750091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5845d-2c19-404c-8e14-63bb0aadaedc admin admin] RBAC: Authorizing `identity:create_grant(project_id=64528a378a7b4ed58d17564e043593d3, user_id=6a95fd43d54e4cd0986b5ea2394e6e69, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:23.753106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5845d-2c19-404c-8e14-63bb0aadaedc admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:23.764114 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1307/2620] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:40:23 2022] PUT /identity/v3/projects/64528a378a7b4ed58d17564e043593d3/users/6a95fd43d54e4cd0986b5ea2394e6e69/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:25.108714 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a98c47-5675-4cae-a016-89046c493a78 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:25.109180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a98c47-5675-4cae-a016-89046c493a78 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:25.109294 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a98c47-5675-4cae-a016-89046c493a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:25.139632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-d9a98c47-5675-4cae-a016-89046c493a78 None None] MFA Rules not processed for user `6a95fd43d54e4cd0986b5ea2394e6e69`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:25.140887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d9a98c47-5675-4cae-a016-89046c493a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:25.217398 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1314/2621] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:25 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:26.573881 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5845d-2c19-404c-8e14-63bb0aadaedc admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:26.644541 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5845d-2c19-404c-8e14-63bb0aadaedc admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:26.649506 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5310d0d-44aa-4026-b82c-af0dd9c85998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:26.650436 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5310d0d-44aa-4026-b82c-af0dd9c85998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:26.650727 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5310d0d-44aa-4026-b82c-af0dd9c85998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:26.651101 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5310d0d-44aa-4026-b82c-af0dd9c85998 admin admin] PATH_INFO: `/v3/users/8c97b76fc0394966a76eaad5b30bc496` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:26.652724 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5310d0d-44aa-4026-b82c-af0dd9c85998 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c97b76fc0394966a76eaad5b30bc496)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:26.654291 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5310d0d-44aa-4026-b82c-af0dd9c85998 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:26.863012 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e5310d0d-44aa-4026-b82c-af0dd9c85998 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c97b76fc0394966a76eaad5b30bc496'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:26.986394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e5310d0d-44aa-4026-b82c-af0dd9c85998 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c97b76fc0394966a76eaad5b30bc496'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:26.987086 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e5310d0d-44aa-4026-b82c-af0dd9c85998 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c97b76fc0394966a76eaad5b30bc496'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:27.000690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e5310d0d-44aa-4026-b82c-af0dd9c85998 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c97b76fc0394966a76eaad5b30bc496'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:27.007582 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e5310d0d-44aa-4026-b82c-af0dd9c85998 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c97b76fc0394966a76eaad5b30bc496'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:27.009769 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:27.014823 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1308/2622] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:40:26 2022] DELETE /identity/v3/users/8c97b76fc0394966a76eaad5b30bc496 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:27.311318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a98c47-5675-4cae-a016-89046c493a78 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:27.322557 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a98c47-5675-4cae-a016-89046c493a78 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:27.392680 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:27.393483 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:27.393768 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:27.394046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe admin admin] PATH_INFO: `/v3/projects/39ad4f66e6434dc98537140e19974111` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:27.395287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=39ad4f66e6434dc98537140e19974111)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:27.397202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:27.575024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39ad4f66e6434dc98537140e19974111', 'request_id': 'req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:27.578972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:27.648935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39ad4f66e6434dc98537140e19974111', 'request_id': 'req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:27.655490 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39ad4f66e6434dc98537140e19974111', 'request_id': 'req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:27.666438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe admin admin] The token cache is being invalidate because project 39ad4f66e6434dc98537140e19974111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:40:27.673277 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:27.673277 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:27.673277 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1315/2623] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:40:27 2022] DELETE /identity/v3/projects/39ad4f66e6434dc98537140e19974111 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:27.673635 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5310d0d-44aa-4026-b82c-af0dd9c85998 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:27.674932 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e5310d0d-44aa-4026-b82c-af0dd9c85998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:27.688471 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5310d0d-44aa-4026-b82c-af0dd9c85998 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:27.743705 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df2951-7ac3-4a70-b3ec-a9dff35a92fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:27.744569 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df2951-7ac3-4a70-b3ec-a9dff35a92fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:27.744847 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df2951-7ac3-4a70-b3ec-a9dff35a92fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:27.745117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df2951-7ac3-4a70-b3ec-a9dff35a92fa admin admin] PATH_INFO: `/v3/users/6a95fd43d54e4cd0986b5ea2394e6e69` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:27.746325 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df2951-7ac3-4a70-b3ec-a9dff35a92fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a95fd43d54e4cd0986b5ea2394e6e69)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:27.748253 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df2951-7ac3-4a70-b3ec-a9dff35a92fa admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:27.943680 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e7df2951-7ac3-4a70-b3ec-a9dff35a92fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a95fd43d54e4cd0986b5ea2394e6e69'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:28.003073 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e7df2951-7ac3-4a70-b3ec-a9dff35a92fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a95fd43d54e4cd0986b5ea2394e6e69'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:28.003698 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e7df2951-7ac3-4a70-b3ec-a9dff35a92fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a95fd43d54e4cd0986b5ea2394e6e69'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:28.016820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e7df2951-7ac3-4a70-b3ec-a9dff35a92fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a95fd43d54e4cd0986b5ea2394e6e69'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:28.023535 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e7df2951-7ac3-4a70-b3ec-a9dff35a92fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a95fd43d54e4cd0986b5ea2394e6e69'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:28.033982 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1309/2624] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:40:27 2022] DELETE /identity/v3/users/6a95fd43d54e4cd0986b5ea2394e6e69 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:28.324772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:28.336353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:28.341340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d36ea-5476-45c0-acfa-ed5e638b97fe admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:28.403205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2d7d5-2968-4cdb-a19e-d828906a02a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:28.405348 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2d7d5-2968-4cdb-a19e-d828906a02a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:28.405348 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2d7d5-2968-4cdb-a19e-d828906a02a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:28.405560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2d7d5-2968-4cdb-a19e-d828906a02a8 admin admin] PATH_INFO: `/v3/projects/64528a378a7b4ed58d17564e043593d3` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:28.406663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2d7d5-2968-4cdb-a19e-d828906a02a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64528a378a7b4ed58d17564e043593d3)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:28.408575 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2d7d5-2968-4cdb-a19e-d828906a02a8 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:28.519743 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-69e2d7d5-2968-4cdb-a19e-d828906a02a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64528a378a7b4ed58d17564e043593d3', 'request_id': 'req-69e2d7d5-2968-4cdb-a19e-d828906a02a8'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:28.626010 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-69e2d7d5-2968-4cdb-a19e-d828906a02a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64528a378a7b4ed58d17564e043593d3', 'request_id': 'req-69e2d7d5-2968-4cdb-a19e-d828906a02a8'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:28.633045 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-69e2d7d5-2968-4cdb-a19e-d828906a02a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64528a378a7b4ed58d17564e043593d3', 'request_id': 'req-69e2d7d5-2968-4cdb-a19e-d828906a02a8'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:28.644292 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-69e2d7d5-2968-4cdb-a19e-d828906a02a8 admin admin] The token cache is being invalidate because project 64528a378a7b4ed58d17564e043593d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:40:28.644585 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-69e2d7d5-2968-4cdb-a19e-d828906a02a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:28.645229 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-69e2d7d5-2968-4cdb-a19e-d828906a02a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:28.647002 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1316/2625] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:40:28 2022] DELETE /identity/v3/projects/64528a378a7b4ed58d17564e043593d3 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:28.651873 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17e28a-ec34-4f49-8970-89fc316f511a None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:28.652309 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17e28a-ec34-4f49-8970-89fc316f511a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:28.652652 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17e28a-ec34-4f49-8970-89fc316f511a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:28.671894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-fd17e28a-ec34-4f49-8970-89fc316f511a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:28.673203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-fd17e28a-ec34-4f49-8970-89fc316f511a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:28.734829 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1310/2626] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:40:28.742825 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885ca31-32c8-4e59-92b2-63c749f8f674 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:28.743111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885ca31-32c8-4e59-92b2-63c749f8f674 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:28.743413 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885ca31-32c8-4e59-92b2-63c749f8f674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:28.762415 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-3885ca31-32c8-4e59-92b2-63c749f8f674 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:28.763614 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-3885ca31-32c8-4e59-92b2-63c749f8f674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:28.772527 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1317/2627] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:28.776922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17e28a-ec34-4f49-8970-89fc316f511a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:28.786944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17e28a-ec34-4f49-8970-89fc316f511a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:28.791577 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa5a6c-919b-45ff-b6a7-0d1c8cf94911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:28.792237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa5a6c-919b-45ff-b6a7-0d1c8cf94911 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:28.792556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa5a6c-919b-45ff-b6a7-0d1c8cf94911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:28.792793 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa5a6c-919b-45ff-b6a7-0d1c8cf94911 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:28.793593 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa5a6c-919b-45ff-b6a7-0d1c8cf94911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:40:28.793906 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa5a6c-919b-45ff-b6a7-0d1c8cf94911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:28.795569 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa5a6c-919b-45ff-b6a7-0d1c8cf94911 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:28.807689 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1311/2628] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:40:28.812816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885ca31-32c8-4e59-92b2-63c749f8f674 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:28.822527 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885ca31-32c8-4e59-92b2-63c749f8f674 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:28.828010 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bcb57-909b-41c4-a119-5d373f82c28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:28.828949 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bcb57-909b-41c4-a119-5d373f82c28d admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:28.829237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bcb57-909b-41c4-a119-5d373f82c28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:28.829513 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bcb57-909b-41c4-a119-5d373f82c28d admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:28.830215 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5bcb57-909b-41c4-a119-5d373f82c28d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:28.832202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5bcb57-909b-41c4-a119-5d373f82c28d admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:28.973528 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1318/2629] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:40:28 2022] POST /identity/v3/projects => generated 387 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:40:28.974844 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa5a6c-919b-45ff-b6a7-0d1c8cf94911 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:28.986090 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa5a6c-919b-45ff-b6a7-0d1c8cf94911 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:28.989874 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:29.044496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a719ca-e1da-4eef-9b1e-61af933b3b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:29.045671 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a719ca-e1da-4eef-9b1e-61af933b3b00 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:29.045952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a719ca-e1da-4eef-9b1e-61af933b3b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:29.046225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a719ca-e1da-4eef-9b1e-61af933b3b00 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:29.046984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a719ca-e1da-4eef-9b1e-61af933b3b00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:29.048943 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a719ca-e1da-4eef-9b1e-61af933b3b00 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:29.169524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1312/2630] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:40:28 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:40:29.175058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bcb57-909b-41c4-a119-5d373f82c28d admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:29.187496 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bcb57-909b-41c4-a119-5d373f82c28d admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:29.192511 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0834ff-3028-4212-bd7d-64142ecf56f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:29.193184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0834ff-3028-4212-bd7d-64142ecf56f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:29.193466 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0834ff-3028-4212-bd7d-64142ecf56f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:29.193775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0834ff-3028-4212-bd7d-64142ecf56f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:29.194538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0834ff-3028-4212-bd7d-64142ecf56f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:29.196148 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0834ff-3028-4212-bd7d-64142ecf56f1 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:29.206990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1319/2631] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:40:29.211445 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a719ca-e1da-4eef-9b1e-61af933b3b00 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:29.227685 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a719ca-e1da-4eef-9b1e-61af933b3b00 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:29.235004 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972ce01-4015-499d-8d91-ab903df2d40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:29.235745 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972ce01-4015-499d-8d91-ab903df2d40e admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:29.236032 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972ce01-4015-499d-8d91-ab903df2d40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:29.236331 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972ce01-4015-499d-8d91-ab903df2d40e admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:29.237097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972ce01-4015-499d-8d91-ab903df2d40e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:29.238757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972ce01-4015-499d-8d91-ab903df2d40e admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:29.249351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1313/2632] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:40:29.254378 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0834ff-3028-4212-bd7d-64142ecf56f1 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:29.264824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0834ff-3028-4212-bd7d-64142ecf56f1 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:29.269838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55843954-90c6-448b-8338-0f0adc73368e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:29.270830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55843954-90c6-448b-8338-0f0adc73368e admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:29.274250 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55843954-90c6-448b-8338-0f0adc73368e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:29.274250 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55843954-90c6-448b-8338-0f0adc73368e admin admin] PATH_INFO: `/v3/projects/d0c2d9d6db584b339fa73aa989d33dd6/users/35170d7875aa47e8b872d0e1ca3328b6/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:29.287072 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55843954-90c6-448b-8338-0f0adc73368e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0c2d9d6db584b339fa73aa989d33dd6, user_id=35170d7875aa47e8b872d0e1ca3328b6, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:29.289714 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55843954-90c6-448b-8338-0f0adc73368e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:29.398381 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1320/2633] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:40:29 2022] PUT /identity/v3/projects/d0c2d9d6db584b339fa73aa989d33dd6/users/35170d7875aa47e8b872d0e1ca3328b6/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:29.409788 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8a851-10fe-4bf3-9001-3888ec5fd4ee None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:29.410081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8a851-10fe-4bf3-9001-3888ec5fd4ee None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:29.410393 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8a851-10fe-4bf3-9001-3888ec5fd4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:29.437511 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-d9e8a851-10fe-4bf3-9001-3888ec5fd4ee None None] MFA Rules not processed for user `35170d7875aa47e8b872d0e1ca3328b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:29.438849 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d9e8a851-10fe-4bf3-9001-3888ec5fd4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:29.511886 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1314/2634] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:29.516934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55843954-90c6-448b-8338-0f0adc73368e admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:29.532945 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:29.606686 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55843954-90c6-448b-8338-0f0adc73368e admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:29.612533 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068d6ae-954e-4e73-9fb6-9a9a3f88720d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:29.613327 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068d6ae-954e-4e73-9fb6-9a9a3f88720d admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:29.613670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068d6ae-954e-4e73-9fb6-9a9a3f88720d admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:29.614012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068d6ae-954e-4e73-9fb6-9a9a3f88720d admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:29.614865 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2068d6ae-954e-4e73-9fb6-9a9a3f88720d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:29.616851 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2068d6ae-954e-4e73-9fb6-9a9a3f88720d admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:29.743258 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1321/2635] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:40:29 2022] POST /identity/v3/projects => generated 385 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:40:29.747012 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8a851-10fe-4bf3-9001-3888ec5fd4ee None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:29.759282 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8a851-10fe-4bf3-9001-3888ec5fd4ee None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:29.828545 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888158a-b57e-4b80-b99d-f62690b4130f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:29.829425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888158a-b57e-4b80-b99d-f62690b4130f admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:29.829712 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888158a-b57e-4b80-b99d-f62690b4130f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:29.830007 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888158a-b57e-4b80-b99d-f62690b4130f admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:29.830804 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888158a-b57e-4b80-b99d-f62690b4130f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:29.832819 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888158a-b57e-4b80-b99d-f62690b4130f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:29.936717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1315/2636] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:40:29 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:40:29.940702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068d6ae-954e-4e73-9fb6-9a9a3f88720d admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:29.956353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068d6ae-954e-4e73-9fb6-9a9a3f88720d admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:29.956645 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654fb63-d369-44f9-9aad-31d2ccdb9561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:29.957327 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654fb63-d369-44f9-9aad-31d2ccdb9561 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:29.957604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654fb63-d369-44f9-9aad-31d2ccdb9561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:29.957882 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654fb63-d369-44f9-9aad-31d2ccdb9561 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:29.958648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654fb63-d369-44f9-9aad-31d2ccdb9561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:29.960247 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654fb63-d369-44f9-9aad-31d2ccdb9561 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:29.971641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1322/2637] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:40:29.975819 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888158a-b57e-4b80-b99d-f62690b4130f admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:29.982618 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:29.986700 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888158a-b57e-4b80-b99d-f62690b4130f admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:29.992024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333a4d1-5ac6-4e5d-b117-23c1114d863f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:29.992930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333a4d1-5ac6-4e5d-b117-23c1114d863f admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:29.993217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333a4d1-5ac6-4e5d-b117-23c1114d863f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:29.993491 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333a4d1-5ac6-4e5d-b117-23c1114d863f admin admin] PATH_INFO: `/v3/projects/4d9c3b4f22ae4121a484ba6278312644/users/dc6c10d8ffdb4c599eb06c3040362ad1/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:30.009061 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333a4d1-5ac6-4e5d-b117-23c1114d863f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d9c3b4f22ae4121a484ba6278312644, user_id=dc6c10d8ffdb4c599eb06c3040362ad1, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:30.011902 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333a4d1-5ac6-4e5d-b117-23c1114d863f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:30.089410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1316/2638] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:40:29 2022] PUT /identity/v3/projects/4d9c3b4f22ae4121a484ba6278312644/users/dc6c10d8ffdb4c599eb06c3040362ad1/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:30.093492 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654fb63-d369-44f9-9aad-31d2ccdb9561 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:30.104185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654fb63-d369-44f9-9aad-31d2ccdb9561 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:30.163561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9adece7-3acc-4041-a7a2-488c70e7176a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:30.164276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adece7-3acc-4041-a7a2-488c70e7176a admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:30.165242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adece7-3acc-4041-a7a2-488c70e7176a admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:30.165524 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adece7-3acc-4041-a7a2-488c70e7176a admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:30.166290 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9adece7-3acc-4041-a7a2-488c70e7176a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:30.167902 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9adece7-3acc-4041-a7a2-488c70e7176a admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:30.181191 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1323/2639] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:40:30.182784 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333a4d1-5ac6-4e5d-b117-23c1114d863f admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:30.195671 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333a4d1-5ac6-4e5d-b117-23c1114d863f admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:30.200501 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f567096-8163-40c6-86d2-284edc2ad2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:30.202160 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f567096-8163-40c6-86d2-284edc2ad2bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:30.202543 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f567096-8163-40c6-86d2-284edc2ad2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:30.202889 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f567096-8163-40c6-86d2-284edc2ad2bc admin admin] PATH_INFO: `/v3/projects/4d9c3b4f22ae4121a484ba6278312644/users/dc6c10d8ffdb4c599eb06c3040362ad1/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:30.205278 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f567096-8163-40c6-86d2-284edc2ad2bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d9c3b4f22ae4121a484ba6278312644, user_id=dc6c10d8ffdb4c599eb06c3040362ad1, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:30.208209 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f567096-8163-40c6-86d2-284edc2ad2bc admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:30.220092 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1317/2640] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:40:30 2022] PUT /identity/v3/projects/4d9c3b4f22ae4121a484ba6278312644/users/dc6c10d8ffdb4c599eb06c3040362ad1/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:30.230654 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32816083-c219-4e6a-bd22-782432e007f4 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:30.231055 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32816083-c219-4e6a-bd22-782432e007f4 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:30.231237 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32816083-c219-4e6a-bd22-782432e007f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:30.259045 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-32816083-c219-4e6a-bd22-782432e007f4 None None] MFA Rules not processed for user `dc6c10d8ffdb4c599eb06c3040362ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:30.260567 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-32816083-c219-4e6a-bd22-782432e007f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:30.331800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1324/2641] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:30.343696 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f567096-8163-40c6-86d2-284edc2ad2bc admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:30.345483 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0f567096-8163-40c6-86d2-284edc2ad2bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:30.414915 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f567096-8163-40c6-86d2-284edc2ad2bc admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:30.419834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484db74-8b25-4b14-9cda-60152cefa2b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:30.420081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484db74-8b25-4b14-9cda-60152cefa2b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:30.420332 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484db74-8b25-4b14-9cda-60152cefa2b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:30.420609 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484db74-8b25-4b14-9cda-60152cefa2b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:30.431907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484db74-8b25-4b14-9cda-60152cefa2b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:30.433619 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484db74-8b25-4b14-9cda-60152cefa2b0 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:30.444378 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1318/2642] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:40:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:40:30.515862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32816083-c219-4e6a-bd22-782432e007f4 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:30.521733 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:30.582671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32816083-c219-4e6a-bd22-782432e007f4 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:30.587550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa5355-4ba6-4c48-b4be-e7022f4e85e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:30.588422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa5355-4ba6-4c48-b4be-e7022f4e85e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:30.588705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa5355-4ba6-4c48-b4be-e7022f4e85e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:30.588984 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa5355-4ba6-4c48-b4be-e7022f4e85e5 admin admin] PATH_INFO: `/v3/users/35170d7875aa47e8b872d0e1ca3328b6` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:30.590212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa5355-4ba6-4c48-b4be-e7022f4e85e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35170d7875aa47e8b872d0e1ca3328b6)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:30.592769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa5355-4ba6-4c48-b4be-e7022f4e85e5 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:30.800126 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-92fa5355-4ba6-4c48-b4be-e7022f4e85e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35170d7875aa47e8b872d0e1ca3328b6'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:30.865053 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-92fa5355-4ba6-4c48-b4be-e7022f4e85e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35170d7875aa47e8b872d0e1ca3328b6'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:30.873836 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-92fa5355-4ba6-4c48-b4be-e7022f4e85e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35170d7875aa47e8b872d0e1ca3328b6'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:30.875139 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-92fa5355-4ba6-4c48-b4be-e7022f4e85e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35170d7875aa47e8b872d0e1ca3328b6'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:30.887328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-92fa5355-4ba6-4c48-b4be-e7022f4e85e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35170d7875aa47e8b872d0e1ca3328b6'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:30.898400 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1325/2643] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:40:30 2022] DELETE /identity/v3/users/35170d7875aa47e8b872d0e1ca3328b6 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:30.899383 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e7794-8448-4690-bb8c-739e7222b383 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:30.899815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e7794-8448-4690-bb8c-739e7222b383 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:30.900319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e7794-8448-4690-bb8c-739e7222b383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:30.917834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-478e7794-8448-4690-bb8c-739e7222b383 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:30.919074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-478e7794-8448-4690-bb8c-739e7222b383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:30.983381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1319/2644] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:30.992981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa5355-4ba6-4c48-b4be-e7022f4e85e5 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:30.994587 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-92fa5355-4ba6-4c48-b4be-e7022f4e85e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:31.053470 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa5355-4ba6-4c48-b4be-e7022f4e85e5 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:31.058698 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a4f34-81c2-4ced-8396-cb9f9f238e59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:31.059530 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a4f34-81c2-4ced-8396-cb9f9f238e59 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:31.059919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a4f34-81c2-4ced-8396-cb9f9f238e59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:31.060284 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a4f34-81c2-4ced-8396-cb9f9f238e59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:31.070958 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a4f34-81c2-4ced-8396-cb9f9f238e59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:31.072517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a4f34-81c2-4ced-8396-cb9f9f238e59 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:31.085529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1326/2645] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:40:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:40:31.473848 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e7794-8448-4690-bb8c-739e7222b383 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:31.475366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-478e7794-8448-4690-bb8c-739e7222b383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:31.487646 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e7794-8448-4690-bb8c-739e7222b383 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:31.492538 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571469aa-8583-44c4-a3cd-4c930e0b4ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:31.493560 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571469aa-8583-44c4-a3cd-4c930e0b4ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:31.493882 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571469aa-8583-44c4-a3cd-4c930e0b4ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:31.494217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571469aa-8583-44c4-a3cd-4c930e0b4ec4 admin admin] PATH_INFO: `/v3/projects/d0c2d9d6db584b339fa73aa989d33dd6` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:31.497223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571469aa-8583-44c4-a3cd-4c930e0b4ec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0c2d9d6db584b339fa73aa989d33dd6)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:31.499642 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571469aa-8583-44c4-a3cd-4c930e0b4ec4 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:31.655070 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-571469aa-8583-44c4-a3cd-4c930e0b4ec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0c2d9d6db584b339fa73aa989d33dd6', 'request_id': 'req-571469aa-8583-44c4-a3cd-4c930e0b4ec4'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:31.717124 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-571469aa-8583-44c4-a3cd-4c930e0b4ec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0c2d9d6db584b339fa73aa989d33dd6', 'request_id': 'req-571469aa-8583-44c4-a3cd-4c930e0b4ec4'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:31.724054 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-571469aa-8583-44c4-a3cd-4c930e0b4ec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0c2d9d6db584b339fa73aa989d33dd6', 'request_id': 'req-571469aa-8583-44c4-a3cd-4c930e0b4ec4'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:31.728307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:31.735440 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-571469aa-8583-44c4-a3cd-4c930e0b4ec4 admin admin] The token cache is being invalidate because project d0c2d9d6db584b339fa73aa989d33dd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:40:31.735634 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-571469aa-8583-44c4-a3cd-4c930e0b4ec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:31.736335 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-571469aa-8583-44c4-a3cd-4c930e0b4ec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:31.737866 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1320/2646] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:40:31 2022] DELETE /identity/v3/projects/d0c2d9d6db584b339fa73aa989d33dd6 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:31.742113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a4f34-81c2-4ced-8396-cb9f9f238e59 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:31.759371 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a4f34-81c2-4ced-8396-cb9f9f238e59 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:31.779666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:31.813007 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0470d1e-7f27-4d4d-b9fd-26ba4d2e12f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:31.813879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0470d1e-7f27-4d4d-b9fd-26ba4d2e12f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:31.814183 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0470d1e-7f27-4d4d-b9fd-26ba4d2e12f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:31.814481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0470d1e-7f27-4d4d-b9fd-26ba4d2e12f9 admin admin] PATH_INFO: `/v3/users/dc6c10d8ffdb4c599eb06c3040362ad1` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:31.815681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0470d1e-7f27-4d4d-b9fd-26ba4d2e12f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc6c10d8ffdb4c599eb06c3040362ad1)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:31.817581 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0470d1e-7f27-4d4d-b9fd-26ba4d2e12f9 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:32.083344 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f0470d1e-7f27-4d4d-b9fd-26ba4d2e12f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc6c10d8ffdb4c599eb06c3040362ad1'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:32.180336 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f0470d1e-7f27-4d4d-b9fd-26ba4d2e12f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc6c10d8ffdb4c599eb06c3040362ad1'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:32.188364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f0470d1e-7f27-4d4d-b9fd-26ba4d2e12f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc6c10d8ffdb4c599eb06c3040362ad1'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:32.191412 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f0470d1e-7f27-4d4d-b9fd-26ba4d2e12f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc6c10d8ffdb4c599eb06c3040362ad1'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:32.201814 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f0470d1e-7f27-4d4d-b9fd-26ba4d2e12f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc6c10d8ffdb4c599eb06c3040362ad1'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:32.208093 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1327/2647] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:40:31 2022] DELETE /identity/v3/users/dc6c10d8ffdb4c599eb06c3040362ad1 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:32.645482 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571469aa-8583-44c4-a3cd-4c930e0b4ec4 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:32.649241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-571469aa-8583-44c4-a3cd-4c930e0b4ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:32.662552 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571469aa-8583-44c4-a3cd-4c930e0b4ec4 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:32.722596 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc5d16-b9e9-4cdb-ac64-390e7e55c7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:32.723451 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc5d16-b9e9-4cdb-ac64-390e7e55c7a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:32.723734 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc5d16-b9e9-4cdb-ac64-390e7e55c7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:32.724008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc5d16-b9e9-4cdb-ac64-390e7e55c7a8 admin admin] PATH_INFO: `/v3/projects/4d9c3b4f22ae4121a484ba6278312644` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:32.725199 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc5d16-b9e9-4cdb-ac64-390e7e55c7a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d9c3b4f22ae4121a484ba6278312644)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:32.727139 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc5d16-b9e9-4cdb-ac64-390e7e55c7a8 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:32.855359 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e6bc5d16-b9e9-4cdb-ac64-390e7e55c7a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d9c3b4f22ae4121a484ba6278312644', 'request_id': 'req-e6bc5d16-b9e9-4cdb-ac64-390e7e55c7a8'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:32.951929 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e6bc5d16-b9e9-4cdb-ac64-390e7e55c7a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d9c3b4f22ae4121a484ba6278312644', 'request_id': 'req-e6bc5d16-b9e9-4cdb-ac64-390e7e55c7a8'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:32.959969 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e6bc5d16-b9e9-4cdb-ac64-390e7e55c7a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d9c3b4f22ae4121a484ba6278312644', 'request_id': 'req-e6bc5d16-b9e9-4cdb-ac64-390e7e55c7a8'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:32.970899 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e6bc5d16-b9e9-4cdb-ac64-390e7e55c7a8 admin admin] The token cache is being invalidate because project 4d9c3b4f22ae4121a484ba6278312644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:40:32.971187 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e6bc5d16-b9e9-4cdb-ac64-390e7e55c7a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:32.971837 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e6bc5d16-b9e9-4cdb-ac64-390e7e55c7a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:32.973384 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1321/2648] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:40:32 2022] DELETE /identity/v3/projects/4d9c3b4f22ae4121a484ba6278312644 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:32.978325 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291d508-2e85-4d81-ae2d-24d6eaede1e3 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:32.978832 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291d508-2e85-4d81-ae2d-24d6eaede1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:32.979034 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291d508-2e85-4d81-ae2d-24d6eaede1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:32.998190 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-1291d508-2e85-4d81-ae2d-24d6eaede1e3 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:32.999543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1291d508-2e85-4d81-ae2d-24d6eaede1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:33.060645 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1328/2649] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:33.068622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd7d84-34d2-4bcb-890a-64df580151ac None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:33.068913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd7d84-34d2-4bcb-890a-64df580151ac None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:33.069187 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd7d84-34d2-4bcb-890a-64df580151ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:33.088054 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-b3fd7d84-34d2-4bcb-890a-64df580151ac None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:33.093297 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b3fd7d84-34d2-4bcb-890a-64df580151ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:33.103625 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1322/2650] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:33.104999 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291d508-2e85-4d81-ae2d-24d6eaede1e3 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:33.116238 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291d508-2e85-4d81-ae2d-24d6eaede1e3 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:33.121842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff8b07-e110-4848-b218-76de5bfbbb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:33.122550 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff8b07-e110-4848-b218-76de5bfbbb69 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:33.122841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff8b07-e110-4848-b218-76de5bfbbb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:33.123167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff8b07-e110-4848-b218-76de5bfbbb69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:33.123903 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff8b07-e110-4848-b218-76de5bfbbb69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:40:33.124206 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff8b07-e110-4848-b218-76de5bfbbb69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:33.125804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff8b07-e110-4848-b218-76de5bfbbb69 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:33.142422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1329/2651] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:40:33.146733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd7d84-34d2-4bcb-890a-64df580151ac None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:33.156860 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd7d84-34d2-4bcb-890a-64df580151ac None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:33.161593 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea5453-f070-41f9-a846-7eb769059450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:33.162248 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea5453-f070-41f9-a846-7eb769059450 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:33.162558 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea5453-f070-41f9-a846-7eb769059450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:33.162836 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea5453-f070-41f9-a846-7eb769059450 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:33.163545 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea5453-f070-41f9-a846-7eb769059450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:33.165562 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea5453-f070-41f9-a846-7eb769059450 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:33.302050 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1323/2652] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:40:33 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:40:33.307067 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff8b07-e110-4848-b218-76de5bfbbb69 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:33.317059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff8b07-e110-4848-b218-76de5bfbbb69 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:33.348932 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:33.375890 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3638c3-ac30-44b7-a7f2-9d43c6d96a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:33.376599 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3638c3-ac30-44b7-a7f2-9d43c6d96a0e admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:33.376887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3638c3-ac30-44b7-a7f2-9d43c6d96a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:33.377165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3638c3-ac30-44b7-a7f2-9d43c6d96a0e admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:33.377912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3638c3-ac30-44b7-a7f2-9d43c6d96a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:33.379892 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3638c3-ac30-44b7-a7f2-9d43c6d96a0e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:33.531643 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1330/2653] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:40:33 2022] POST /identity/v3/users => generated 287 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:40:33.535717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea5453-f070-41f9-a846-7eb769059450 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:33.545965 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea5453-f070-41f9-a846-7eb769059450 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:33.550702 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ab40c-72bf-4a67-a27b-3088d57782b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:33.552136 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ab40c-72bf-4a67-a27b-3088d57782b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:33.552424 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ab40c-72bf-4a67-a27b-3088d57782b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:33.552707 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ab40c-72bf-4a67-a27b-3088d57782b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:33.553474 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ab40c-72bf-4a67-a27b-3088d57782b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:33.555128 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ab40c-72bf-4a67-a27b-3088d57782b2 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:33.566926 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1324/2654] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:40:33.570538 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3638c3-ac30-44b7-a7f2-9d43c6d96a0e admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:33.581777 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3638c3-ac30-44b7-a7f2-9d43c6d96a0e admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:33.586847 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2f297-673d-4b25-8e86-b428c8798b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:33.587518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2f297-673d-4b25-8e86-b428c8798b49 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:33.587800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2f297-673d-4b25-8e86-b428c8798b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:33.588075 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2f297-673d-4b25-8e86-b428c8798b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:33.588815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2f297-673d-4b25-8e86-b428c8798b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:33.590662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2f297-673d-4b25-8e86-b428c8798b49 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:33.605639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1331/2655] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:40:33.609202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ab40c-72bf-4a67-a27b-3088d57782b2 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:33.617254 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ab40c-72bf-4a67-a27b-3088d57782b2 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:33.622221 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41dd47-4450-45cb-8b52-aeb1cdb25f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:33.623223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41dd47-4450-45cb-8b52-aeb1cdb25f47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:33.623447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41dd47-4450-45cb-8b52-aeb1cdb25f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:33.623726 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41dd47-4450-45cb-8b52-aeb1cdb25f47 admin admin] PATH_INFO: `/v3/projects/a7ba6cd3c6444913af3b9719e1b4224f/users/cfea0653196c4f408cb805e7b6180835/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:33.639742 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41dd47-4450-45cb-8b52-aeb1cdb25f47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7ba6cd3c6444913af3b9719e1b4224f, user_id=cfea0653196c4f408cb805e7b6180835, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:33.642575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41dd47-4450-45cb-8b52-aeb1cdb25f47 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:33.719444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1325/2656] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:40:33 2022] PUT /identity/v3/projects/a7ba6cd3c6444913af3b9719e1b4224f/users/cfea0653196c4f408cb805e7b6180835/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:33.731254 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e247e6-17cb-4f88-8746-f5d04fb955a2 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:33.731544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e247e6-17cb-4f88-8746-f5d04fb955a2 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:33.731821 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e247e6-17cb-4f88-8746-f5d04fb955a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:33.758977 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-47e247e6-17cb-4f88-8746-f5d04fb955a2 None None] MFA Rules not processed for user `cfea0653196c4f408cb805e7b6180835`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:33.760199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-47e247e6-17cb-4f88-8746-f5d04fb955a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:33.825840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1332/2657] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:33.837241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41dd47-4450-45cb-8b52-aeb1cdb25f47 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:33.839206 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4f41dd47-4450-45cb-8b52-aeb1cdb25f47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:33.860381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:33.894979 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41dd47-4450-45cb-8b52-aeb1cdb25f47 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:33.898593 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a8280-2271-4a59-beb0-71d9ac6037ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:33.899233 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a8280-2271-4a59-beb0-71d9ac6037ef service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:33.899514 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a8280-2271-4a59-beb0-71d9ac6037ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:33.899788 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a8280-2271-4a59-beb0-71d9ac6037ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:33.907884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a8280-2271-4a59-beb0-71d9ac6037ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:33.909287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a8280-2271-4a59-beb0-71d9ac6037ef service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:33.918624 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1326/2658] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:40:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:40:38.652435 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e247e6-17cb-4f88-8746-f5d04fb955a2 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:38.717493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e247e6-17cb-4f88-8746-f5d04fb955a2 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:38.721977 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143998d5-eaff-4624-b1ba-c599dcfce5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:38.722896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143998d5-eaff-4624-b1ba-c599dcfce5f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:38.723176 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143998d5-eaff-4624-b1ba-c599dcfce5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:38.723449 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143998d5-eaff-4624-b1ba-c599dcfce5f1 admin admin] PATH_INFO: `/v3/users/cfea0653196c4f408cb805e7b6180835` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:38.724644 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143998d5-eaff-4624-b1ba-c599dcfce5f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfea0653196c4f408cb805e7b6180835)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:38.726603 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143998d5-eaff-4624-b1ba-c599dcfce5f1 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:39.012508 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-143998d5-eaff-4624-b1ba-c599dcfce5f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfea0653196c4f408cb805e7b6180835'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:39.015266 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:39.105785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-143998d5-eaff-4624-b1ba-c599dcfce5f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfea0653196c4f408cb805e7b6180835'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:39.114060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-143998d5-eaff-4624-b1ba-c599dcfce5f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfea0653196c4f408cb805e7b6180835'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:39.114824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-143998d5-eaff-4624-b1ba-c599dcfce5f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfea0653196c4f408cb805e7b6180835'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:39.126468 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-143998d5-eaff-4624-b1ba-c599dcfce5f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfea0653196c4f408cb805e7b6180835'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:39.133840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1333/2659] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:40:38 2022] DELETE /identity/v3/users/cfea0653196c4f408cb805e7b6180835 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:40:39.137346 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c43766-f266-4102-b03c-c3fca323b324 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:39.137986 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c43766-f266-4102-b03c-c3fca323b324 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:39.138424 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c43766-f266-4102-b03c-c3fca323b324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:39.158716 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-85c43766-f266-4102-b03c-c3fca323b324 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:39.160049 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-85c43766-f266-4102-b03c-c3fca323b324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:39.227516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1327/2660] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:40:39.238115 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143998d5-eaff-4624-b1ba-c599dcfce5f1 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:39.244289 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-143998d5-eaff-4624-b1ba-c599dcfce5f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:39.303944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143998d5-eaff-4624-b1ba-c599dcfce5f1 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:39.308120 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db85bb-9436-4b0d-883b-d6e06241d7aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:39.308899 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db85bb-9436-4b0d-883b-d6e06241d7aa service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:39.309174 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db85bb-9436-4b0d-883b-d6e06241d7aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:39.309444 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db85bb-9436-4b0d-883b-d6e06241d7aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:39.319313 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db85bb-9436-4b0d-883b-d6e06241d7aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:39.320731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db85bb-9436-4b0d-883b-d6e06241d7aa service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:39.330366 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1334/2661] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:40:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:40:39.819081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c43766-f266-4102-b03c-c3fca323b324 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:39.820661 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-85c43766-f266-4102-b03c-c3fca323b324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:39.836430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c43766-f266-4102-b03c-c3fca323b324 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:39.841180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0517e-4b94-48e2-899f-53ba6846e86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:39.842090 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0517e-4b94-48e2-899f-53ba6846e86c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:39.842414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0517e-4b94-48e2-899f-53ba6846e86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:39.842688 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0517e-4b94-48e2-899f-53ba6846e86c admin admin] PATH_INFO: `/v3/projects/a7ba6cd3c6444913af3b9719e1b4224f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:39.843881 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0517e-4b94-48e2-899f-53ba6846e86c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7ba6cd3c6444913af3b9719e1b4224f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:39.845825 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0517e-4b94-48e2-899f-53ba6846e86c admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:40.003387 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-03c0517e-4b94-48e2-899f-53ba6846e86c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7ba6cd3c6444913af3b9719e1b4224f', 'request_id': 'req-03c0517e-4b94-48e2-899f-53ba6846e86c'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:40.105674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-03c0517e-4b94-48e2-899f-53ba6846e86c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7ba6cd3c6444913af3b9719e1b4224f', 'request_id': 'req-03c0517e-4b94-48e2-899f-53ba6846e86c'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:40.113253 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-03c0517e-4b94-48e2-899f-53ba6846e86c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7ba6cd3c6444913af3b9719e1b4224f', 'request_id': 'req-03c0517e-4b94-48e2-899f-53ba6846e86c'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:40.123640 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-03c0517e-4b94-48e2-899f-53ba6846e86c admin admin] The token cache is being invalidate because project a7ba6cd3c6444913af3b9719e1b4224f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:40:40.123934 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-03c0517e-4b94-48e2-899f-53ba6846e86c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:40.124579 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-03c0517e-4b94-48e2-899f-53ba6846e86c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:40.126137 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1328/2662] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:40:39 2022] DELETE /identity/v3/projects/a7ba6cd3c6444913af3b9719e1b4224f => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:40.131213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd6380-b1a6-401f-89f0-25fce415fbb5 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:40.131718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd6380-b1a6-401f-89f0-25fce415fbb5 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:40.131827 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd6380-b1a6-401f-89f0-25fce415fbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:40.151185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-d6dd6380-b1a6-401f-89f0-25fce415fbb5 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:40.152377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d6dd6380-b1a6-401f-89f0-25fce415fbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:40.216250 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1335/2663] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:40.218076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00803444-9679-4c1c-832b-12bca3ec1e93 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:40.218591 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00803444-9679-4c1c-832b-12bca3ec1e93 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:40.218862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00803444-9679-4c1c-832b-12bca3ec1e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:40.238293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-00803444-9679-4c1c-832b-12bca3ec1e93 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:40.239499 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-00803444-9679-4c1c-832b-12bca3ec1e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:40.247495 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1329/2664] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:40:40.260385 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd6380-b1a6-401f-89f0-25fce415fbb5 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:40.269951 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd6380-b1a6-401f-89f0-25fce415fbb5 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:40.274450 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069468f8-2826-4478-a460-df7c0d274ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:40.275100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069468f8-2826-4478-a460-df7c0d274ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:40.275456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069468f8-2826-4478-a460-df7c0d274ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:40.275649 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069468f8-2826-4478-a460-df7c0d274ee1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:40.276371 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069468f8-2826-4478-a460-df7c0d274ee1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:40:40.276680 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069468f8-2826-4478-a460-df7c0d274ee1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:40.279190 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069468f8-2826-4478-a460-df7c0d274ee1 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:40.290755 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1336/2665] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:40:40.295640 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00803444-9679-4c1c-832b-12bca3ec1e93 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:40.305778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00803444-9679-4c1c-832b-12bca3ec1e93 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:40.312033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece7ada-0f56-4a8b-9a12-e374eb8f8afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:40.312728 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece7ada-0f56-4a8b-9a12-e374eb8f8afd admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:40.313015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece7ada-0f56-4a8b-9a12-e374eb8f8afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:40.313294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece7ada-0f56-4a8b-9a12-e374eb8f8afd admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:40.314016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece7ada-0f56-4a8b-9a12-e374eb8f8afd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:40.315972 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece7ada-0f56-4a8b-9a12-e374eb8f8afd admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:40.519717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1330/2666] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:40:40 2022] POST /identity/v3/projects => generated 385 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:40:40.523792 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069468f8-2826-4478-a460-df7c0d274ee1 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:40.533965 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069468f8-2826-4478-a460-df7c0d274ee1 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:40.582863 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:40.592544 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454a25a-9a3c-407d-bbd4-f7043e227ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:40.593249 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454a25a-9a3c-407d-bbd4-f7043e227ec0 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:40.593540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454a25a-9a3c-407d-bbd4-f7043e227ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:40.593816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454a25a-9a3c-407d-bbd4-f7043e227ec0 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:40.594554 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454a25a-9a3c-407d-bbd4-f7043e227ec0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:40.596495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454a25a-9a3c-407d-bbd4-f7043e227ec0 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:40.714740 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1337/2667] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:40:40 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:40:40.716005 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece7ada-0f56-4a8b-9a12-e374eb8f8afd admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:40.727555 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece7ada-0f56-4a8b-9a12-e374eb8f8afd admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:40.732643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fd266-5e58-48cf-9259-7febdfdc1729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:40.733522 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fd266-5e58-48cf-9259-7febdfdc1729 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:40.733807 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fd266-5e58-48cf-9259-7febdfdc1729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:40.734097 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fd266-5e58-48cf-9259-7febdfdc1729 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:40.734992 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5fd266-5e58-48cf-9259-7febdfdc1729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:40.744974 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5fd266-5e58-48cf-9259-7febdfdc1729 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:40.748215 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1331/2668] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:40:40.756331 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454a25a-9a3c-407d-bbd4-f7043e227ec0 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:40.767431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454a25a-9a3c-407d-bbd4-f7043e227ec0 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:40.772065 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c63f8-406b-48e4-80b8-40395cbe694e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:40.772672 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c63f8-406b-48e4-80b8-40395cbe694e admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:40.772953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c63f8-406b-48e4-80b8-40395cbe694e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:40.773231 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c63f8-406b-48e4-80b8-40395cbe694e admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:40.773959 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c63f8-406b-48e4-80b8-40395cbe694e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:40.775616 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c63f8-406b-48e4-80b8-40395cbe694e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:40.786046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1338/2669] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:40:40.790220 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fd266-5e58-48cf-9259-7febdfdc1729 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:40.801468 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fd266-5e58-48cf-9259-7febdfdc1729 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:40.814286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe48043-63f0-46b7-93e8-f0b4aa402926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:40.814286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe48043-63f0-46b7-93e8-f0b4aa402926 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:40.814286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe48043-63f0-46b7-93e8-f0b4aa402926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:40.814286 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe48043-63f0-46b7-93e8-f0b4aa402926 admin admin] PATH_INFO: `/v3/projects/84e131e7ae914fd792c150bc50af3e39/users/65b34869d78b4b65a7269a002a78950e/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:40.814792 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:40.824745 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe48043-63f0-46b7-93e8-f0b4aa402926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e131e7ae914fd792c150bc50af3e39, user_id=65b34869d78b4b65a7269a002a78950e, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:40.828045 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe48043-63f0-46b7-93e8-f0b4aa402926 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:40.924294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1332/2670] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:40:40 2022] PUT /identity/v3/projects/84e131e7ae914fd792c150bc50af3e39/users/65b34869d78b4b65a7269a002a78950e/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:40.934718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584c56a-cbaf-465c-90c0-92ffa5a74bab None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:40.935006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584c56a-cbaf-465c-90c0-92ffa5a74bab None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:40.935921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584c56a-cbaf-465c-90c0-92ffa5a74bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:40.961722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-2584c56a-cbaf-465c-90c0-92ffa5a74bab None None] MFA Rules not processed for user `65b34869d78b4b65a7269a002a78950e`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:40.962963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2584c56a-cbaf-465c-90c0-92ffa5a74bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:41.029346 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1339/2671] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:41.033752 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe48043-63f0-46b7-93e8-f0b4aa402926 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:41.104324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe48043-63f0-46b7-93e8-f0b4aa402926 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:41.109674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1d33f-dfff-49b3-90b4-d7119c95b138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:41.115835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1d33f-dfff-49b3-90b4-d7119c95b138 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:41.115835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1d33f-dfff-49b3-90b4-d7119c95b138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:41.115835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1d33f-dfff-49b3-90b4-d7119c95b138 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:41.115835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1d33f-dfff-49b3-90b4-d7119c95b138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:41.115835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1d33f-dfff-49b3-90b4-d7119c95b138 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:41.210246 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1333/2672] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:40:41 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:40:41.214406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584c56a-cbaf-465c-90c0-92ffa5a74bab None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:41.225664 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584c56a-cbaf-465c-90c0-92ffa5a74bab None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:41.285169 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556752f5-3d63-4841-8177-13fceb06eb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:41.285764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556752f5-3d63-4841-8177-13fceb06eb89 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:41.286040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556752f5-3d63-4841-8177-13fceb06eb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:41.286342 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556752f5-3d63-4841-8177-13fceb06eb89 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:41.287032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556752f5-3d63-4841-8177-13fceb06eb89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:41.288931 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556752f5-3d63-4841-8177-13fceb06eb89 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:41.398752 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1340/2673] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:40:41 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:40:41.402712 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1d33f-dfff-49b3-90b4-d7119c95b138 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:41.413495 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1d33f-dfff-49b3-90b4-d7119c95b138 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:41.419149 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ec048-9e3e-49de-ac2f-7bd279ed0197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:41.420410 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ec048-9e3e-49de-ac2f-7bd279ed0197 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:41.420815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ec048-9e3e-49de-ac2f-7bd279ed0197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:41.421101 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ec048-9e3e-49de-ac2f-7bd279ed0197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:41.421862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ec048-9e3e-49de-ac2f-7bd279ed0197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:41.423517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ec048-9e3e-49de-ac2f-7bd279ed0197 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:41.434813 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1334/2674] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:40:41.438903 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556752f5-3d63-4841-8177-13fceb06eb89 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:41.449156 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556752f5-3d63-4841-8177-13fceb06eb89 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:41.454534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509dd77-5d2b-4ff4-8f2d-5acc1dc573c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:41.454896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509dd77-5d2b-4ff4-8f2d-5acc1dc573c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:41.455178 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509dd77-5d2b-4ff4-8f2d-5acc1dc573c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:41.455453 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509dd77-5d2b-4ff4-8f2d-5acc1dc573c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:41.458607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509dd77-5d2b-4ff4-8f2d-5acc1dc573c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:41.458607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509dd77-5d2b-4ff4-8f2d-5acc1dc573c8 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:41.468863 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1341/2675] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:40:41.473212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ec048-9e3e-49de-ac2f-7bd279ed0197 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:41.484060 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ec048-9e3e-49de-ac2f-7bd279ed0197 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:41.488642 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa235b-5a03-434c-bfbb-1e5e1918da50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:41.489518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa235b-5a03-434c-bfbb-1e5e1918da50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:41.489921 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa235b-5a03-434c-bfbb-1e5e1918da50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:41.490208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa235b-5a03-434c-bfbb-1e5e1918da50 admin admin] PATH_INFO: `/v3/projects/c81fccde59f04956a91e495814f54517/users/1894d0fe69b5481cb5544b34c62ac34d/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:41.505996 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa235b-5a03-434c-bfbb-1e5e1918da50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c81fccde59f04956a91e495814f54517, user_id=1894d0fe69b5481cb5544b34c62ac34d, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:41.508684 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa235b-5a03-434c-bfbb-1e5e1918da50 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:41.592210 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1335/2676] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:40:41 2022] PUT /identity/v3/projects/c81fccde59f04956a91e495814f54517/users/1894d0fe69b5481cb5544b34c62ac34d/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:41.603822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c81bf-8268-4e58-9180-d3892d69424e None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:41.604162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c81bf-8268-4e58-9180-d3892d69424e None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:41.604484 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c81bf-8268-4e58-9180-d3892d69424e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:41.632285 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-7b0c81bf-8268-4e58-9180-d3892d69424e None None] MFA Rules not processed for user `1894d0fe69b5481cb5544b34c62ac34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:41.633605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-7b0c81bf-8268-4e58-9180-d3892d69424e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:41.661477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:41.701728 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1342/2677] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:40:41.706604 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa235b-5a03-434c-bfbb-1e5e1918da50 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:41.773275 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa235b-5a03-434c-bfbb-1e5e1918da50 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:41.778573 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6855b45-a233-4893-bc5a-746d86c7d09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:41.779948 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6855b45-a233-4893-bc5a-746d86c7d09c admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:41.780306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6855b45-a233-4893-bc5a-746d86c7d09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:41.780674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6855b45-a233-4893-bc5a-746d86c7d09c admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:41.781430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6855b45-a233-4893-bc5a-746d86c7d09c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:41.783394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6855b45-a233-4893-bc5a-746d86c7d09c admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:41.956593 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1336/2678] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:40:41 2022] POST /identity/v3/projects => generated 385 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:40:41.957668 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c81bf-8268-4e58-9180-d3892d69424e None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:41.969295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c81bf-8268-4e58-9180-d3892d69424e None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:42.027314 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9e824-4ff9-4e0d-84b9-e246b3f7ed47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:42.027997 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9e824-4ff9-4e0d-84b9-e246b3f7ed47 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:42.028284 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9e824-4ff9-4e0d-84b9-e246b3f7ed47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:42.028561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9e824-4ff9-4e0d-84b9-e246b3f7ed47 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:42.029304 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9e824-4ff9-4e0d-84b9-e246b3f7ed47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:42.031359 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9e824-4ff9-4e0d-84b9-e246b3f7ed47 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:42.150152 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1343/2679] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:40:41 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:40:42.151755 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6855b45-a233-4893-bc5a-746d86c7d09c admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:42.162799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6855b45-a233-4893-bc5a-746d86c7d09c admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:42.167672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccffc68-cafb-4df6-b958-78e9540bb728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:42.168358 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccffc68-cafb-4df6-b958-78e9540bb728 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:42.168689 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccffc68-cafb-4df6-b958-78e9540bb728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:42.168959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccffc68-cafb-4df6-b958-78e9540bb728 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:42.169685 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccffc68-cafb-4df6-b958-78e9540bb728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:42.171380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccffc68-cafb-4df6-b958-78e9540bb728 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:42.183012 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1337/2680] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:40:42.192397 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9e824-4ff9-4e0d-84b9-e246b3f7ed47 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:42.203674 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9e824-4ff9-4e0d-84b9-e246b3f7ed47 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:42.208512 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075d892-c701-46a2-8e04-861b9c24b185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:42.209343 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075d892-c701-46a2-8e04-861b9c24b185 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:42.209626 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075d892-c701-46a2-8e04-861b9c24b185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:42.209903 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075d892-c701-46a2-8e04-861b9c24b185 admin admin] PATH_INFO: `/v3/projects/8f6fe353822c4de9b0fbb55fecf96ff2/users/df32a181ff01461086fdc10f5ba0b504/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:42.226532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075d892-c701-46a2-8e04-861b9c24b185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6fe353822c4de9b0fbb55fecf96ff2, user_id=df32a181ff01461086fdc10f5ba0b504, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:42.229202 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075d892-c701-46a2-8e04-861b9c24b185 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:42.309169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926239a3-d05a-4630-892f-93034a289c33 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:42.309465 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926239a3-d05a-4630-892f-93034a289c33 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:42.309749 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926239a3-d05a-4630-892f-93034a289c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:42.322041 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:42.327679 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-926239a3-d05a-4630-892f-93034a289c33 None None] MFA Rules not processed for user `0235b8fe92554131ba30e1d18f0f2d86`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:42.329172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-926239a3-d05a-4630-892f-93034a289c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:42.335495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1344/2681] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:40:42 2022] PUT /identity/v3/projects/8f6fe353822c4de9b0fbb55fecf96ff2/users/df32a181ff01461086fdc10f5ba0b504/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:42.344169 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075d892-c701-46a2-8e04-861b9c24b185 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:42.386443 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1338/2682] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:40:42.409077 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075d892-c701-46a2-8e04-861b9c24b185 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:42.414408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f95699-0708-47ab-bade-34b4f9d81423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:42.415084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f95699-0708-47ab-bade-34b4f9d81423 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:42.415368 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f95699-0708-47ab-bade-34b4f9d81423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:42.415648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f95699-0708-47ab-bade-34b4f9d81423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:42.416372 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f95699-0708-47ab-bade-34b4f9d81423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:42.418069 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f95699-0708-47ab-bade-34b4f9d81423 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:42.434727 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1345/2683] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:40:42.438583 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926239a3-d05a-4630-892f-93034a289c33 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:42.453512 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926239a3-d05a-4630-892f-93034a289c33 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:42.459584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77541c-8319-4fb1-b835-cf4ce1422867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:42.460611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77541c-8319-4fb1-b835-cf4ce1422867 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:42.460906 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77541c-8319-4fb1-b835-cf4ce1422867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:42.461185 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77541c-8319-4fb1-b835-cf4ce1422867 admin admin] PATH_INFO: `/v3/projects/8f6fe353822c4de9b0fbb55fecf96ff2/users/df32a181ff01461086fdc10f5ba0b504/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:42.463385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77541c-8319-4fb1-b835-cf4ce1422867 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6fe353822c4de9b0fbb55fecf96ff2, user_id=df32a181ff01461086fdc10f5ba0b504, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:42.466109 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77541c-8319-4fb1-b835-cf4ce1422867 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:42.478089 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1339/2684] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:40:42 2022] PUT /identity/v3/projects/8f6fe353822c4de9b0fbb55fecf96ff2/users/df32a181ff01461086fdc10f5ba0b504/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:42.488767 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd3474-5ae9-4014-92f9-4a6f2a19bcad None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:42.489379 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd3474-5ae9-4014-92f9-4a6f2a19bcad None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:42.489653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd3474-5ae9-4014-92f9-4a6f2a19bcad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:42.521885 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-aefd3474-5ae9-4014-92f9-4a6f2a19bcad None None] MFA Rules not processed for user `df32a181ff01461086fdc10f5ba0b504`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:42.523222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-aefd3474-5ae9-4014-92f9-4a6f2a19bcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:42.541386 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:42.601379 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1346/2685] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:40:42.613108 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77541c-8319-4fb1-b835-cf4ce1422867 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:42.615793 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-aa77541c-8319-4fb1-b835-cf4ce1422867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:42.677476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77541c-8319-4fb1-b835-cf4ce1422867 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:42.681275 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7195b5-9896-4125-9c06-429452ed977d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:42.681930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7195b5-9896-4125-9c06-429452ed977d service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:42.682212 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7195b5-9896-4125-9c06-429452ed977d service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:42.682519 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7195b5-9896-4125-9c06-429452ed977d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:42.742021 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7195b5-9896-4125-9c06-429452ed977d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:42.743417 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7195b5-9896-4125-9c06-429452ed977d service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:42.753319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1340/2686] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:40:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:40:45.181994 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefd3474-5ae9-4014-92f9-4a6f2a19bcad None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:45.183439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-aefd3474-5ae9-4014-92f9-4a6f2a19bcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:45.195981 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefd3474-5ae9-4014-92f9-4a6f2a19bcad None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:45.199584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208a16b-0a21-480c-857e-644b0fd443d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:45.200375 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208a16b-0a21-480c-857e-644b0fd443d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:45.200663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208a16b-0a21-480c-857e-644b0fd443d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:45.201540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208a16b-0a21-480c-857e-644b0fd443d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:45.257996 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208a16b-0a21-480c-857e-644b0fd443d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:45.259487 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208a16b-0a21-480c-857e-644b0fd443d7 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:45.269060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1347/2687] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:40:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:40:47.079287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7195b5-9896-4125-9c06-429452ed977d service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:47.088151 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7195b5-9896-4125-9c06-429452ed977d service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:47.092133 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303e674-0759-4bbb-8966-64a741ff3b3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:47.092815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303e674-0759-4bbb-8966-64a741ff3b3f service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:47.093098 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303e674-0759-4bbb-8966-64a741ff3b3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:47.093375 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303e674-0759-4bbb-8966-64a741ff3b3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:47.104260 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303e674-0759-4bbb-8966-64a741ff3b3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:47.105717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303e674-0759-4bbb-8966-64a741ff3b3f service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:47.118470 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1341/2688] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:40:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:40:52.126805 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208a16b-0a21-480c-857e-644b0fd443d7 service cinder] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:52.187964 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208a16b-0a21-480c-857e-644b0fd443d7 service cinder] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:52.192502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cad1f-f5d9-49a0-a284-2c7956222b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:52.193372 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cad1f-f5d9-49a0-a284-2c7956222b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:52.193655 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cad1f-f5d9-49a0-a284-2c7956222b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:52.193933 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cad1f-f5d9-49a0-a284-2c7956222b76 admin admin] PATH_INFO: `/v3/users/65b34869d78b4b65a7269a002a78950e` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:52.195218 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9cad1f-f5d9-49a0-a284-2c7956222b76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65b34869d78b4b65a7269a002a78950e)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:52.197146 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9cad1f-f5d9-49a0-a284-2c7956222b76 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:52.428168 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-3c9cad1f-f5d9-49a0-a284-2c7956222b76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65b34869d78b4b65a7269a002a78950e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:52.540714 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-3c9cad1f-f5d9-49a0-a284-2c7956222b76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65b34869d78b4b65a7269a002a78950e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:52.542531 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:52.549400 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-3c9cad1f-f5d9-49a0-a284-2c7956222b76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65b34869d78b4b65a7269a002a78950e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:52.551314 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-3c9cad1f-f5d9-49a0-a284-2c7956222b76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65b34869d78b4b65a7269a002a78950e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:52.570391 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-3c9cad1f-f5d9-49a0-a284-2c7956222b76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65b34869d78b4b65a7269a002a78950e'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:52.577796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1348/2689] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:40:52 2022] DELETE /identity/v3/users/65b34869d78b4b65a7269a002a78950e => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:52.583550 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c708ae-8819-4b98-bae9-2506795938ba None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:52.584145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c708ae-8819-4b98-bae9-2506795938ba None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:52.584262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c708ae-8819-4b98-bae9-2506795938ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:52.603135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-c0c708ae-8819-4b98-bae9-2506795938ba None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:52.604385 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-c0c708ae-8819-4b98-bae9-2506795938ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:52.646574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:52.671826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1342/2690] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:40:52.682205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cad1f-f5d9-49a0-a284-2c7956222b76 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:52.686699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-3c9cad1f-f5d9-49a0-a284-2c7956222b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:52.740876 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cad1f-f5d9-49a0-a284-2c7956222b76 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:52.744434 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ffaaf-52b8-41e5-9060-5447d4524448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:52.745074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ffaaf-52b8-41e5-9060-5447d4524448 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:52.745353 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ffaaf-52b8-41e5-9060-5447d4524448 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:52.745625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ffaaf-52b8-41e5-9060-5447d4524448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:52.754911 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ffaaf-52b8-41e5-9060-5447d4524448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:52.756331 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ffaaf-52b8-41e5-9060-5447d4524448 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:52.766585 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1349/2691] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:40:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:40:53.469111 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c708ae-8819-4b98-bae9-2506795938ba None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:53.470653 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-c0c708ae-8819-4b98-bae9-2506795938ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:53.484258 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c708ae-8819-4b98-bae9-2506795938ba None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:53.489542 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6b773-57e4-4de4-aca9-f9138e177d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:53.490531 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6b773-57e4-4de4-aca9-f9138e177d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:53.490820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6b773-57e4-4de4-aca9-f9138e177d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:53.491099 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6b773-57e4-4de4-aca9-f9138e177d6e admin admin] PATH_INFO: `/v3/projects/84e131e7ae914fd792c150bc50af3e39` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:53.492364 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6b773-57e4-4de4-aca9-f9138e177d6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=84e131e7ae914fd792c150bc50af3e39)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:53.497988 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6b773-57e4-4de4-aca9-f9138e177d6e admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:53.662842 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ded6b773-57e4-4de4-aca9-f9138e177d6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84e131e7ae914fd792c150bc50af3e39', 'request_id': 'req-ded6b773-57e4-4de4-aca9-f9138e177d6e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:53.780462 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ded6b773-57e4-4de4-aca9-f9138e177d6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84e131e7ae914fd792c150bc50af3e39', 'request_id': 'req-ded6b773-57e4-4de4-aca9-f9138e177d6e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:53.787662 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ded6b773-57e4-4de4-aca9-f9138e177d6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84e131e7ae914fd792c150bc50af3e39', 'request_id': 'req-ded6b773-57e4-4de4-aca9-f9138e177d6e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:53.798316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ded6b773-57e4-4de4-aca9-f9138e177d6e admin admin] The token cache is being invalidate because project 84e131e7ae914fd792c150bc50af3e39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:40:53.798608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ded6b773-57e4-4de4-aca9-f9138e177d6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:53.799672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ded6b773-57e4-4de4-aca9-f9138e177d6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:53.802289 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1343/2692] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:40:53 2022] DELETE /identity/v3/projects/84e131e7ae914fd792c150bc50af3e39 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:53.806199 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ffaaf-52b8-41e5-9060-5447d4524448 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:53.816309 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ffaaf-52b8-41e5-9060-5447d4524448 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:53.882659 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcaecf-3449-415b-9a2e-546e9836fb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:53.883618 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcaecf-3449-415b-9a2e-546e9836fb90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:53.883907 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcaecf-3449-415b-9a2e-546e9836fb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:53.884185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcaecf-3449-415b-9a2e-546e9836fb90 admin admin] PATH_INFO: `/v3/users/1894d0fe69b5481cb5544b34c62ac34d` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:53.885396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcaecf-3449-415b-9a2e-546e9836fb90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1894d0fe69b5481cb5544b34c62ac34d)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:53.887435 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcaecf-3449-415b-9a2e-546e9836fb90 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:54.189040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-4bfcaecf-3449-415b-9a2e-546e9836fb90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1894d0fe69b5481cb5544b34c62ac34d'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:54.311704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-4bfcaecf-3449-415b-9a2e-546e9836fb90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1894d0fe69b5481cb5544b34c62ac34d'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:54.322055 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-4bfcaecf-3449-415b-9a2e-546e9836fb90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1894d0fe69b5481cb5544b34c62ac34d'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:54.323005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-4bfcaecf-3449-415b-9a2e-546e9836fb90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1894d0fe69b5481cb5544b34c62ac34d'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:54.336697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:54.341449 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-4bfcaecf-3449-415b-9a2e-546e9836fb90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1894d0fe69b5481cb5544b34c62ac34d'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:54.354699 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1350/2693] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:40:53 2022] DELETE /identity/v3/users/1894d0fe69b5481cb5544b34c62ac34d => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:54.910382 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6b773-57e4-4de4-aca9-f9138e177d6e admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:54.913579 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ded6b773-57e4-4de4-aca9-f9138e177d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:54.928865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6b773-57e4-4de4-aca9-f9138e177d6e admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:55.011736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5048952-7adb-4d34-8b0a-dfa354b3677f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:55.012763 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5048952-7adb-4d34-8b0a-dfa354b3677f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:55.013917 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5048952-7adb-4d34-8b0a-dfa354b3677f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:55.014208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5048952-7adb-4d34-8b0a-dfa354b3677f admin admin] PATH_INFO: `/v3/projects/c81fccde59f04956a91e495814f54517` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:55.015622 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5048952-7adb-4d34-8b0a-dfa354b3677f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c81fccde59f04956a91e495814f54517)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:55.017581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5048952-7adb-4d34-8b0a-dfa354b3677f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:55.134562 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e5048952-7adb-4d34-8b0a-dfa354b3677f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c81fccde59f04956a91e495814f54517', 'request_id': 'req-e5048952-7adb-4d34-8b0a-dfa354b3677f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:55.288032 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e5048952-7adb-4d34-8b0a-dfa354b3677f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c81fccde59f04956a91e495814f54517', 'request_id': 'req-e5048952-7adb-4d34-8b0a-dfa354b3677f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:55.298881 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e5048952-7adb-4d34-8b0a-dfa354b3677f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c81fccde59f04956a91e495814f54517', 'request_id': 'req-e5048952-7adb-4d34-8b0a-dfa354b3677f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:55.303299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:55.310566 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e5048952-7adb-4d34-8b0a-dfa354b3677f admin admin] The token cache is being invalidate because project c81fccde59f04956a91e495814f54517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:40:55.310857 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e5048952-7adb-4d34-8b0a-dfa354b3677f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:55.311538 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-e5048952-7adb-4d34-8b0a-dfa354b3677f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:55.313100 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1344/2694] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:40:54 2022] DELETE /identity/v3/projects/c81fccde59f04956a91e495814f54517 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:55.317151 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcaecf-3449-415b-9a2e-546e9836fb90 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:55.318615 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-4bfcaecf-3449-415b-9a2e-546e9836fb90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:55.332144 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcaecf-3449-415b-9a2e-546e9836fb90 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:55.389417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1157bf-4ee5-4e5d-b0d7-b521baf89a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:55.390511 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1157bf-4ee5-4e5d-b0d7-b521baf89a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:55.390798 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1157bf-4ee5-4e5d-b0d7-b521baf89a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:55.391068 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1157bf-4ee5-4e5d-b0d7-b521baf89a21 admin admin] PATH_INFO: `/v3/users/df32a181ff01461086fdc10f5ba0b504` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:55.392408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1157bf-4ee5-4e5d-b0d7-b521baf89a21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df32a181ff01461086fdc10f5ba0b504)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:55.395088 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1157bf-4ee5-4e5d-b0d7-b521baf89a21 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:55.615633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7a1157bf-4ee5-4e5d-b0d7-b521baf89a21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df32a181ff01461086fdc10f5ba0b504'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:55.699396 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7a1157bf-4ee5-4e5d-b0d7-b521baf89a21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df32a181ff01461086fdc10f5ba0b504'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:55.708112 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7a1157bf-4ee5-4e5d-b0d7-b521baf89a21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df32a181ff01461086fdc10f5ba0b504'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:55.708753 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7a1157bf-4ee5-4e5d-b0d7-b521baf89a21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df32a181ff01461086fdc10f5ba0b504'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:55.722081 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-7a1157bf-4ee5-4e5d-b0d7-b521baf89a21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df32a181ff01461086fdc10f5ba0b504'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:55.727996 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1351/2695] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:40:55 2022] DELETE /identity/v3/users/df32a181ff01461086fdc10f5ba0b504 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:56.232524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5048952-7adb-4d34-8b0a-dfa354b3677f admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:56.235223 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e5048952-7adb-4d34-8b0a-dfa354b3677f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:56.246779 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5048952-7adb-4d34-8b0a-dfa354b3677f admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:56.313102 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8ecef-1dd1-4bc4-b436-2f8eff19bec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:56.313966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8ecef-1dd1-4bc4-b436-2f8eff19bec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:56.314279 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8ecef-1dd1-4bc4-b436-2f8eff19bec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:56.314580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8ecef-1dd1-4bc4-b436-2f8eff19bec5 admin admin] PATH_INFO: `/v3/projects/8f6fe353822c4de9b0fbb55fecf96ff2` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:56.315859 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8ecef-1dd1-4bc4-b436-2f8eff19bec5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f6fe353822c4de9b0fbb55fecf96ff2)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:56.317764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8ecef-1dd1-4bc4-b436-2f8eff19bec5 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:56.459904 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-4be8ecef-1dd1-4bc4-b436-2f8eff19bec5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f6fe353822c4de9b0fbb55fecf96ff2', 'request_id': 'req-4be8ecef-1dd1-4bc4-b436-2f8eff19bec5'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:56.545414 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-4be8ecef-1dd1-4bc4-b436-2f8eff19bec5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f6fe353822c4de9b0fbb55fecf96ff2', 'request_id': 'req-4be8ecef-1dd1-4bc4-b436-2f8eff19bec5'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:56.552265 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-4be8ecef-1dd1-4bc4-b436-2f8eff19bec5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f6fe353822c4de9b0fbb55fecf96ff2', 'request_id': 'req-4be8ecef-1dd1-4bc4-b436-2f8eff19bec5'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:56.562572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-4be8ecef-1dd1-4bc4-b436-2f8eff19bec5 admin admin] The token cache is being invalidate because project 8f6fe353822c4de9b0fbb55fecf96ff2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:40:56.562916 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-4be8ecef-1dd1-4bc4-b436-2f8eff19bec5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:56.563501 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-4be8ecef-1dd1-4bc4-b436-2f8eff19bec5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:40:56.565080 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1345/2696] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:40:56 2022] DELETE /identity/v3/projects/8f6fe353822c4de9b0fbb55fecf96ff2 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:56.570000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ce072-b207-49ba-91d5-ef81f156b5c5 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:56.570509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ce072-b207-49ba-91d5-ef81f156b5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:56.570741 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ce072-b207-49ba-91d5-ef81f156b5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:56.589417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-0e4ce072-b207-49ba-91d5-ef81f156b5c5 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:56.590705 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0e4ce072-b207-49ba-91d5-ef81f156b5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:56.651540 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1352/2697] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:56.661472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6941a-2987-434c-b4e6-00b68ab64a6f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:56.661771 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6941a-2987-434c-b4e6-00b68ab64a6f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:56.662053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6941a-2987-434c-b4e6-00b68ab64a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:56.681456 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-25a6941a-2987-434c-b4e6-00b68ab64a6f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:56.682701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-25a6941a-2987-434c-b4e6-00b68ab64a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:56.690828 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1346/2698] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:56.695117 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ce072-b207-49ba-91d5-ef81f156b5c5 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:56.705784 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ce072-b207-49ba-91d5-ef81f156b5c5 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:56.710380 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511317b-60cc-4f00-a2e7-4736cee3460c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:56.711030 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511317b-60cc-4f00-a2e7-4736cee3460c admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:56.711312 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511317b-60cc-4f00-a2e7-4736cee3460c admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:56.711586 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511317b-60cc-4f00-a2e7-4736cee3460c admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:56.712303 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511317b-60cc-4f00-a2e7-4736cee3460c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:40:56.712617 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511317b-60cc-4f00-a2e7-4736cee3460c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:56.714234 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511317b-60cc-4f00-a2e7-4736cee3460c admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:56.724506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:56.727842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1353/2699] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:40:56.732194 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6941a-2987-434c-b4e6-00b68ab64a6f None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:56.741654 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6941a-2987-434c-b4e6-00b68ab64a6f None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:56.752355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822fcc98-cc03-43ef-983e-11ff2e381e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:56.752355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fcc98-cc03-43ef-983e-11ff2e381e81 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:56.752355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fcc98-cc03-43ef-983e-11ff2e381e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:56.752355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fcc98-cc03-43ef-983e-11ff2e381e81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:56.752768 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822fcc98-cc03-43ef-983e-11ff2e381e81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:56.752768 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822fcc98-cc03-43ef-983e-11ff2e381e81 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:56.836227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1347/2700] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:40:56 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:40:56.840332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511317b-60cc-4f00-a2e7-4736cee3460c admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:56.850945 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511317b-60cc-4f00-a2e7-4736cee3460c admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:56.907829 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4e5a8-e06f-408e-8aa6-67d788537a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:56.908503 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4e5a8-e06f-408e-8aa6-67d788537a6a admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:56.908785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4e5a8-e06f-408e-8aa6-67d788537a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:56.909060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4e5a8-e06f-408e-8aa6-67d788537a6a admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:56.909812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4e5a8-e06f-408e-8aa6-67d788537a6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:56.911728 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4e5a8-e06f-408e-8aa6-67d788537a6a admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:56.996709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1354/2701] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:40:56 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:40:57.001202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822fcc98-cc03-43ef-983e-11ff2e381e81 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:57.011476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822fcc98-cc03-43ef-983e-11ff2e381e81 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:57.016740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103bb91d-4872-4553-85f8-75ab1e5171ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:57.017403 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103bb91d-4872-4553-85f8-75ab1e5171ad admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:57.017766 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103bb91d-4872-4553-85f8-75ab1e5171ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:57.018056 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103bb91d-4872-4553-85f8-75ab1e5171ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:57.018822 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103bb91d-4872-4553-85f8-75ab1e5171ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:57.020416 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103bb91d-4872-4553-85f8-75ab1e5171ad admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:57.031103 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1348/2702] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:40:57.035242 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4e5a8-e06f-408e-8aa6-67d788537a6a admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:57.045457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4e5a8-e06f-408e-8aa6-67d788537a6a admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:57.050125 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301c142-2f69-4992-859d-b65d58dba1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:57.050820 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301c142-2f69-4992-859d-b65d58dba1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:57.051105 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301c142-2f69-4992-859d-b65d58dba1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:57.051491 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301c142-2f69-4992-859d-b65d58dba1e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:57.052218 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301c142-2f69-4992-859d-b65d58dba1e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:57.053799 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301c142-2f69-4992-859d-b65d58dba1e3 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:57.064184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1355/2703] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:40:57.068261 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103bb91d-4872-4553-85f8-75ab1e5171ad admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:57.079519 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103bb91d-4872-4553-85f8-75ab1e5171ad admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:57.084121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f87010-9154-48c4-af58-798a3d387a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:57.085025 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f87010-9154-48c4-af58-798a3d387a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:57.085312 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f87010-9154-48c4-af58-798a3d387a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:57.085584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f87010-9154-48c4-af58-798a3d387a7b admin admin] PATH_INFO: `/v3/projects/093f88479eae442dba4d8320fbcb2b40/users/a86c442d85e34a3487c8ad101045ee54/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:57.102219 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f87010-9154-48c4-af58-798a3d387a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=093f88479eae442dba4d8320fbcb2b40, user_id=a86c442d85e34a3487c8ad101045ee54, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:57.104959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f87010-9154-48c4-af58-798a3d387a7b admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:57.197035 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1349/2704] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:40:57 2022] PUT /identity/v3/projects/093f88479eae442dba4d8320fbcb2b40/users/a86c442d85e34a3487c8ad101045ee54/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:40:57.203350 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f6fe2-2837-40be-945d-505aff5b73df None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:57.203802 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f6fe2-2837-40be-945d-505aff5b73df None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:57.204149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f6fe2-2837-40be-945d-505aff5b73df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:57.222494 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-520f6fe2-2837-40be-945d-505aff5b73df None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:57.223702 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-520f6fe2-2837-40be-945d-505aff5b73df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:57.281219 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1356/2705] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:57.291028 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f87010-9154-48c4-af58-798a3d387a7b admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:57.292810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e8f87010-9154-48c4-af58-798a3d387a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:57.315582 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:40:57.352417 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f87010-9154-48c4-af58-798a3d387a7b admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:57.356043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08850a-46ac-4afd-b23b-03a376adc326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:57.357186 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08850a-46ac-4afd-b23b-03a376adc326 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:57.357472 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08850a-46ac-4afd-b23b-03a376adc326 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:57.357753 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08850a-46ac-4afd-b23b-03a376adc326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:57.367716 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08850a-46ac-4afd-b23b-03a376adc326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:57.369151 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08850a-46ac-4afd-b23b-03a376adc326 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:57.386118 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1350/2706] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:40:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:40:58.652248 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8296518-72a8-4d1f-b4e9-46e3c3130738 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:58.652800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8296518-72a8-4d1f-b4e9-46e3c3130738 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:58.653000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8296518-72a8-4d1f-b4e9-46e3c3130738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:58.687269 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-c8296518-72a8-4d1f-b4e9-46e3c3130738 None None] MFA Rules not processed for user `a86c442d85e34a3487c8ad101045ee54`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:40:58.688497 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c8296518-72a8-4d1f-b4e9-46e3c3130738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:58.772918 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1357/2707] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:40:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:40:58.785004 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08850a-46ac-4afd-b23b-03a376adc326 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:40:58.834795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-dd08850a-46ac-4afd-b23b-03a376adc326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:40:58.854582 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08850a-46ac-4afd-b23b-03a376adc326 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:40:58.859691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1e431-aa15-4188-bda2-ff377be530d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:40:58.860363 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1e431-aa15-4188-bda2-ff377be530d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:40:58.860815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1e431-aa15-4188-bda2-ff377be530d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:40:58.861103 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1e431-aa15-4188-bda2-ff377be530d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:40:58.873373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1e431-aa15-4188-bda2-ff377be530d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:40:58.874906 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1e431-aa15-4188-bda2-ff377be530d5 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:40:58.890556 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1351/2708] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:40:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:41:00.677532 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c5069-95fe-4606-a8d6-0a67912b2d60 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:00.678078 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c5069-95fe-4606-a8d6-0a67912b2d60 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:00.678184 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c5069-95fe-4606-a8d6-0a67912b2d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:00.696295 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-961c5069-95fe-4606-a8d6-0a67912b2d60 None None] MFA Rules not processed for user `a86c442d85e34a3487c8ad101045ee54`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:41:00.697502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-961c5069-95fe-4606-a8d6-0a67912b2d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:00.704718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1358/2709] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:41:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:41:03.955424 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1e431-aa15-4188-bda2-ff377be530d5 service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:03.957041 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a1d1e431-aa15-4188-bda2-ff377be530d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:03.969659 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1e431-aa15-4188-bda2-ff377be530d5 service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:03.973262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa305a-70d2-46e3-bdd2-5d370f72cabc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:03.973893 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa305a-70d2-46e3-bdd2-5d370f72cabc service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:03.974172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa305a-70d2-46e3-bdd2-5d370f72cabc service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:03.974490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa305a-70d2-46e3-bdd2-5d370f72cabc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:03.976241 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4cfa305a-70d2-46e3-bdd2-5d370f72cabc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:04.038329 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa305a-70d2-46e3-bdd2-5d370f72cabc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:04.039799 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa305a-70d2-46e3-bdd2-5d370f72cabc service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:04.050473 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1352/2710] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:41:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:41:10.691161 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:41:11.172345 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c5069-95fe-4606-a8d6-0a67912b2d60 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:11.182240 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c5069-95fe-4606-a8d6-0a67912b2d60 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:11.185930 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e487e8-5e39-467b-8819-e6e93a42cffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:11.186618 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e487e8-5e39-467b-8819-e6e93a42cffa service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:11.186905 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e487e8-5e39-467b-8819-e6e93a42cffa service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:11.187209 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e487e8-5e39-467b-8819-e6e93a42cffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:11.188985 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-50e487e8-5e39-467b-8819-e6e93a42cffa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:11.201687 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e487e8-5e39-467b-8819-e6e93a42cffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:11.203437 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e487e8-5e39-467b-8819-e6e93a42cffa service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:11.212982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1359/2711] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:41:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:41:14.045405 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=71 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:41:19.439588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa305a-70d2-46e3-bdd2-5d370f72cabc service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:19.441142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4cfa305a-70d2-46e3-bdd2-5d370f72cabc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:19.455822 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa305a-70d2-46e3-bdd2-5d370f72cabc service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:19.460490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c779358-2448-4cf6-b8c9-22ced1220c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:19.461373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c779358-2448-4cf6-b8c9-22ced1220c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:19.461659 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c779358-2448-4cf6-b8c9-22ced1220c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:19.461966 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c779358-2448-4cf6-b8c9-22ced1220c74 admin admin] PATH_INFO: `/v3/users/ec2e023235a343dd81754211ee102f3c` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:19.463508 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c779358-2448-4cf6-b8c9-22ced1220c74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec2e023235a343dd81754211ee102f3c)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:19.467302 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c779358-2448-4cf6-b8c9-22ced1220c74 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:19.700630 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5c779358-2448-4cf6-b8c9-22ced1220c74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec2e023235a343dd81754211ee102f3c'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:19.790637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5c779358-2448-4cf6-b8c9-22ced1220c74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec2e023235a343dd81754211ee102f3c'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:19.791295 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5c779358-2448-4cf6-b8c9-22ced1220c74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec2e023235a343dd81754211ee102f3c'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:19.804690 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5c779358-2448-4cf6-b8c9-22ced1220c74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec2e023235a343dd81754211ee102f3c'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:19.811817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-5c779358-2448-4cf6-b8c9-22ced1220c74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec2e023235a343dd81754211ee102f3c'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:19.817847 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1353/2712] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:41:19 2022] DELETE /identity/v3/users/ec2e023235a343dd81754211ee102f3c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:41:20.198029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e487e8-5e39-467b-8819-e6e93a42cffa service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:20.214863 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e487e8-5e39-467b-8819-e6e93a42cffa service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:20.281963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165812c8-2233-459b-9de5-93429f45ad57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:20.282879 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165812c8-2233-459b-9de5-93429f45ad57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:20.283161 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165812c8-2233-459b-9de5-93429f45ad57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:20.283430 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165812c8-2233-459b-9de5-93429f45ad57 admin admin] PATH_INFO: `/v3/projects/09d92d320f8748c58aca3fd6a8776540` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:20.284662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165812c8-2233-459b-9de5-93429f45ad57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09d92d320f8748c58aca3fd6a8776540)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:20.286627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165812c8-2233-459b-9de5-93429f45ad57 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:20.427270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-165812c8-2233-459b-9de5-93429f45ad57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09d92d320f8748c58aca3fd6a8776540', 'request_id': 'req-165812c8-2233-459b-9de5-93429f45ad57'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:20.544574 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-165812c8-2233-459b-9de5-93429f45ad57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09d92d320f8748c58aca3fd6a8776540', 'request_id': 'req-165812c8-2233-459b-9de5-93429f45ad57'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:20.551724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-165812c8-2233-459b-9de5-93429f45ad57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09d92d320f8748c58aca3fd6a8776540', 'request_id': 'req-165812c8-2233-459b-9de5-93429f45ad57'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:20.563017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-165812c8-2233-459b-9de5-93429f45ad57 admin admin] The token cache is being invalidate because project 09d92d320f8748c58aca3fd6a8776540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:41:20.563309 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-165812c8-2233-459b-9de5-93429f45ad57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:20.563956 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-165812c8-2233-459b-9de5-93429f45ad57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:20.565521 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1360/2713] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:41:20 2022] DELETE /identity/v3/projects/09d92d320f8748c58aca3fd6a8776540 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:41:20.569487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c779358-2448-4cf6-b8c9-22ced1220c74 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:20.570952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5c779358-2448-4cf6-b8c9-22ced1220c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:20.585202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c779358-2448-4cf6-b8c9-22ced1220c74 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:20.664252 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d47619-193d-47ad-b147-6179fde9a541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:20.665228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d47619-193d-47ad-b147-6179fde9a541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:20.665529 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d47619-193d-47ad-b147-6179fde9a541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:20.665817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d47619-193d-47ad-b147-6179fde9a541 admin admin] PATH_INFO: `/v3/users/a98af3ac148a4c72830f13182195fd13` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:20.667192 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d47619-193d-47ad-b147-6179fde9a541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a98af3ac148a4c72830f13182195fd13)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:20.674506 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d47619-193d-47ad-b147-6179fde9a541 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:20.936481 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-21d47619-193d-47ad-b147-6179fde9a541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a98af3ac148a4c72830f13182195fd13'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:21.058548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-21d47619-193d-47ad-b147-6179fde9a541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a98af3ac148a4c72830f13182195fd13'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:21.059778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-21d47619-193d-47ad-b147-6179fde9a541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a98af3ac148a4c72830f13182195fd13'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:21.073953 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-21d47619-193d-47ad-b147-6179fde9a541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a98af3ac148a4c72830f13182195fd13'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:21.076907 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:41:21.085970 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-21d47619-193d-47ad-b147-6179fde9a541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a98af3ac148a4c72830f13182195fd13'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:21.092918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1354/2714] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:41:20 2022] DELETE /identity/v3/users/a98af3ac148a4c72830f13182195fd13 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:41:21.385950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165812c8-2233-459b-9de5-93429f45ad57 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:21.387568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-165812c8-2233-459b-9de5-93429f45ad57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:21.401083 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165812c8-2233-459b-9de5-93429f45ad57 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:21.459721 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed7d82-9df0-42b0-bdba-a9ad9ab7f3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:21.460505 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed7d82-9df0-42b0-bdba-a9ad9ab7f3d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:21.460783 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed7d82-9df0-42b0-bdba-a9ad9ab7f3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:21.461057 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed7d82-9df0-42b0-bdba-a9ad9ab7f3d9 admin admin] PATH_INFO: `/v3/projects/24edc8866f044562ad23afe2815af7b0` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:21.462280 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed7d82-9df0-42b0-bdba-a9ad9ab7f3d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24edc8866f044562ad23afe2815af7b0)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:21.464201 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed7d82-9df0-42b0-bdba-a9ad9ab7f3d9 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:21.485203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:41:21.625792 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-65ed7d82-9df0-42b0-bdba-a9ad9ab7f3d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24edc8866f044562ad23afe2815af7b0', 'request_id': 'req-65ed7d82-9df0-42b0-bdba-a9ad9ab7f3d9'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:21.749967 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-65ed7d82-9df0-42b0-bdba-a9ad9ab7f3d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24edc8866f044562ad23afe2815af7b0', 'request_id': 'req-65ed7d82-9df0-42b0-bdba-a9ad9ab7f3d9'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:21.756757 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-65ed7d82-9df0-42b0-bdba-a9ad9ab7f3d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24edc8866f044562ad23afe2815af7b0', 'request_id': 'req-65ed7d82-9df0-42b0-bdba-a9ad9ab7f3d9'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:21.767889 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-65ed7d82-9df0-42b0-bdba-a9ad9ab7f3d9 admin admin] The token cache is being invalidate because project 24edc8866f044562ad23afe2815af7b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:41:21.768179 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-65ed7d82-9df0-42b0-bdba-a9ad9ab7f3d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:21.768807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-65ed7d82-9df0-42b0-bdba-a9ad9ab7f3d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:41:21.770369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1361/2715] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:41:21 2022] DELETE /identity/v3/projects/24edc8866f044562ad23afe2815af7b0 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:41:21.775400 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543fd2b-1e2f-497d-8f93-457dabd46154 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:21.775919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543fd2b-1e2f-497d-8f93-457dabd46154 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:21.776043 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543fd2b-1e2f-497d-8f93-457dabd46154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:21.795284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-a543fd2b-1e2f-497d-8f93-457dabd46154 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:41:21.796494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a543fd2b-1e2f-497d-8f93-457dabd46154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:21.856766 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1355/2716] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:41:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:41:21.917656 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aa2cf-6fc3-44cb-9559-7d5cfbaea6ec None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:21.917944 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aa2cf-6fc3-44cb-9559-7d5cfbaea6ec None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:21.918855 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aa2cf-6fc3-44cb-9559-7d5cfbaea6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:21.937406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-1f7aa2cf-6fc3-44cb-9559-7d5cfbaea6ec None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:41:21.938636 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-1f7aa2cf-6fc3-44cb-9559-7d5cfbaea6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:21.946572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1362/2717] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:41:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:41:21.950805 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543fd2b-1e2f-497d-8f93-457dabd46154 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:21.961016 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543fd2b-1e2f-497d-8f93-457dabd46154 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:21.966454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7b952-70ed-461d-81ce-5709d205d708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:21.967115 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7b952-70ed-461d-81ce-5709d205d708 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:21.967401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7b952-70ed-461d-81ce-5709d205d708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:21.967676 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7b952-70ed-461d-81ce-5709d205d708 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:21.968611 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7b952-70ed-461d-81ce-5709d205d708 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:41:21.969053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7b952-70ed-461d-81ce-5709d205d708 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:21.970716 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7b952-70ed-461d-81ce-5709d205d708 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:21.986469 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1356/2718] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:41:21.990637 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7aa2cf-6fc3-44cb-9559-7d5cfbaea6ec None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:22.000825 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7aa2cf-6fc3-44cb-9559-7d5cfbaea6ec None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:22.005477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4def640-cc74-4fae-b0e7-44ab50215132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:22.006147 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4def640-cc74-4fae-b0e7-44ab50215132 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:22.006527 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4def640-cc74-4fae-b0e7-44ab50215132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:22.006813 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4def640-cc74-4fae-b0e7-44ab50215132 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:22.007521 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4def640-cc74-4fae-b0e7-44ab50215132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:22.009432 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4def640-cc74-4fae-b0e7-44ab50215132 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:22.137376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1363/2719] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:41:21 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:41:22.141590 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7b952-70ed-461d-81ce-5709d205d708 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:22.154252 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7b952-70ed-461d-81ce-5709d205d708 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:22.216197 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fe4d5-f567-4c01-9930-d2e8d8df7b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:22.216921 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fe4d5-f567-4c01-9930-d2e8d8df7b73 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:22.217229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fe4d5-f567-4c01-9930-d2e8d8df7b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:22.217494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fe4d5-f567-4c01-9930-d2e8d8df7b73 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:22.218219 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554fe4d5-f567-4c01-9930-d2e8d8df7b73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:22.220216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554fe4d5-f567-4c01-9930-d2e8d8df7b73 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:22.265903 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8279b15-0e2f-4bc6-9626-735edbabd8b8 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:22.266198 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8279b15-0e2f-4bc6-9626-735edbabd8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:22.266537 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8279b15-0e2f-4bc6-9626-735edbabd8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:22.285208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-b8279b15-0e2f-4bc6-9626-735edbabd8b8 None None] MFA Rules not processed for user `a86c442d85e34a3487c8ad101045ee54`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:41:22.286460 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b8279b15-0e2f-4bc6-9626-735edbabd8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:22.343154 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1364/2720] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:41:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:41:22.427345 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1357/2721] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:41:22 2022] POST /identity/v3/users => generated 289 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:41:22.436137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8279b15-0e2f-4bc6-9626-735edbabd8b8 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:22.447517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8279b15-0e2f-4bc6-9626-735edbabd8b8 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:22.453215 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56ed13-c3f9-4c3d-a1c2-b7e792812897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:22.453971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56ed13-c3f9-4c3d-a1c2-b7e792812897 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:22.454420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56ed13-c3f9-4c3d-a1c2-b7e792812897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:22.454701 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56ed13-c3f9-4c3d-a1c2-b7e792812897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:22.455483 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56ed13-c3f9-4c3d-a1c2-b7e792812897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:22.457321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56ed13-c3f9-4c3d-a1c2-b7e792812897 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:22.468328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1365/2722] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:41:22.472517 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fe4d5-f567-4c01-9930-d2e8d8df7b73 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:22.482953 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fe4d5-f567-4c01-9930-d2e8d8df7b73 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:22.487860 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f1290-541c-4682-94e3-a70a5e0fc30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:22.488623 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f1290-541c-4682-94e3-a70a5e0fc30b admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:22.488980 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f1290-541c-4682-94e3-a70a5e0fc30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:22.489326 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f1290-541c-4682-94e3-a70a5e0fc30b admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:22.490127 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f1290-541c-4682-94e3-a70a5e0fc30b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:22.491826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f1290-541c-4682-94e3-a70a5e0fc30b admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:22.502835 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1358/2723] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:41:22.507288 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56ed13-c3f9-4c3d-a1c2-b7e792812897 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:22.518122 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56ed13-c3f9-4c3d-a1c2-b7e792812897 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:22.524402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a476d-a138-4690-982d-4929054ac4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:22.525458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a476d-a138-4690-982d-4929054ac4e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:22.525731 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a476d-a138-4690-982d-4929054ac4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:22.526012 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a476d-a138-4690-982d-4929054ac4e2 admin admin] PATH_INFO: `/v3/projects/68372ee91fb843108d3b0cb1c99cd186/users/cb4dc884b7eb4aa8a0adcdb46c5f9586/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:22.542100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a476d-a138-4690-982d-4929054ac4e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68372ee91fb843108d3b0cb1c99cd186, user_id=cb4dc884b7eb4aa8a0adcdb46c5f9586, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:22.545009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a476d-a138-4690-982d-4929054ac4e2 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:22.649235 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1366/2724] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:41:22 2022] PUT /identity/v3/projects/68372ee91fb843108d3b0cb1c99cd186/users/cb4dc884b7eb4aa8a0adcdb46c5f9586/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:41:22.654817 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc9fa8-33bb-4890-823e-5c42d4c31700 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:22.655095 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc9fa8-33bb-4890-823e-5c42d4c31700 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:22.655381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc9fa8-33bb-4890-823e-5c42d4c31700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:22.674694 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-84dc9fa8-33bb-4890-823e-5c42d4c31700 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:41:22.675952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-84dc9fa8-33bb-4890-823e-5c42d4c31700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:22.732024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:41:22.741445 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1359/2725] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:41:22.752754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a476d-a138-4690-982d-4929054ac4e2 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:22.754539 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-402a476d-a138-4690-982d-4929054ac4e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:22.888467 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:41:22.892098 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a476d-a138-4690-982d-4929054ac4e2 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:22.895898 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678253d6-3e97-442f-b501-f869c5f41951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:22.896580 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678253d6-3e97-442f-b501-f869c5f41951 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:22.896861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678253d6-3e97-442f-b501-f869c5f41951 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:22.897137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678253d6-3e97-442f-b501-f869c5f41951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:22.907016 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678253d6-3e97-442f-b501-f869c5f41951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:22.908465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678253d6-3e97-442f-b501-f869c5f41951 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:22.918953 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1367/2726] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:41:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:41:30.563162 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471203a-6e25-4f65-9d6a-ad6ebe446c1b None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:30.563162 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471203a-6e25-4f65-9d6a-ad6ebe446c1b None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:30.563751 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471203a-6e25-4f65-9d6a-ad6ebe446c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:30.606137 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-0471203a-6e25-4f65-9d6a-ad6ebe446c1b None None] MFA Rules not processed for user `cb4dc884b7eb4aa8a0adcdb46c5f9586`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:41:30.607519 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0471203a-6e25-4f65-9d6a-ad6ebe446c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:30.681642 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1360/2727] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:41:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:41:30.686511 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678253d6-3e97-442f-b501-f869c5f41951 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:30.697412 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678253d6-3e97-442f-b501-f869c5f41951 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:30.703201 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48590e8b-32db-4f21-84fb-6a745901df67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:30.703880 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48590e8b-32db-4f21-84fb-6a745901df67 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:30.704173 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48590e8b-32db-4f21-84fb-6a745901df67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:30.704448 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48590e8b-32db-4f21-84fb-6a745901df67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:30.705155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48590e8b-32db-4f21-84fb-6a745901df67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:30.707213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48590e8b-32db-4f21-84fb-6a745901df67 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:30.849739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1368/2728] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:41:30 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:41:30.855037 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471203a-6e25-4f65-9d6a-ad6ebe446c1b None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:30.867089 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471203a-6e25-4f65-9d6a-ad6ebe446c1b None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:30.959126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdb212-fdbf-43df-a7e5-8e3aa0009d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:30.959126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdb212-fdbf-43df-a7e5-8e3aa0009d50 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:30.959126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdb212-fdbf-43df-a7e5-8e3aa0009d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:30.959126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdb212-fdbf-43df-a7e5-8e3aa0009d50 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:30.959499 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdb212-fdbf-43df-a7e5-8e3aa0009d50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:30.959499 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdb212-fdbf-43df-a7e5-8e3aa0009d50 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:31.081301 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1361/2729] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:41:30 2022] POST /identity/v3/users => generated 295 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:41:31.083840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48590e8b-32db-4f21-84fb-6a745901df67 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:31.111460 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48590e8b-32db-4f21-84fb-6a745901df67 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:31.117793 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be9aa3-f6c1-45dc-bed2-257ebe327074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:31.122438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be9aa3-f6c1-45dc-bed2-257ebe327074 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:31.122438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be9aa3-f6c1-45dc-bed2-257ebe327074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:31.122438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be9aa3-f6c1-45dc-bed2-257ebe327074 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:31.122438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be9aa3-f6c1-45dc-bed2-257ebe327074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:31.122438 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be9aa3-f6c1-45dc-bed2-257ebe327074 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:31.135226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1369/2730] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:41:31.139576 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdb212-fdbf-43df-a7e5-8e3aa0009d50 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:31.152354 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdb212-fdbf-43df-a7e5-8e3aa0009d50 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:31.159317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc896f-954e-4b13-9001-af5418f2ced0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:31.160573 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc896f-954e-4b13-9001-af5418f2ced0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:31.160863 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc896f-954e-4b13-9001-af5418f2ced0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:31.161145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc896f-954e-4b13-9001-af5418f2ced0 admin admin] PATH_INFO: `/v3/projects/980fc587eb054036b5d7cde3f4d2a625/users/8cddba4b99e542499f27e0bf14f012b3/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:31.178686 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc896f-954e-4b13-9001-af5418f2ced0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=980fc587eb054036b5d7cde3f4d2a625, user_id=8cddba4b99e542499f27e0bf14f012b3, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:31.181560 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc896f-954e-4b13-9001-af5418f2ced0 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:31.273412 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1362/2731] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:41:31 2022] PUT /identity/v3/projects/980fc587eb054036b5d7cde3f4d2a625/users/8cddba4b99e542499f27e0bf14f012b3/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:41:31.281076 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be9aa3-f6c1-45dc-bed2-257ebe327074 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:31.292327 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be9aa3-f6c1-45dc-bed2-257ebe327074 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:31.366739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42000c99-d8f2-4e09-a655-2c10630c541e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:31.367455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42000c99-d8f2-4e09-a655-2c10630c541e admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:31.367739 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42000c99-d8f2-4e09-a655-2c10630c541e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:31.368019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42000c99-d8f2-4e09-a655-2c10630c541e admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:31.368776 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42000c99-d8f2-4e09-a655-2c10630c541e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:31.370421 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42000c99-d8f2-4e09-a655-2c10630c541e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:31.381782 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1370/2732] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:41:31.386026 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc896f-954e-4b13-9001-af5418f2ced0 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:31.396494 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc896f-954e-4b13-9001-af5418f2ced0 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:31.401323 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51badda8-dc4a-4125-88ad-5dbc604b59f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:31.401926 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51badda8-dc4a-4125-88ad-5dbc604b59f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:31.402216 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51badda8-dc4a-4125-88ad-5dbc604b59f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:31.402528 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51badda8-dc4a-4125-88ad-5dbc604b59f4 admin admin] PATH_INFO: `/v3/projects/980fc587eb054036b5d7cde3f4d2a625/users/8cddba4b99e542499f27e0bf14f012b3/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:31.404524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51badda8-dc4a-4125-88ad-5dbc604b59f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=980fc587eb054036b5d7cde3f4d2a625, user_id=8cddba4b99e542499f27e0bf14f012b3, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:31.407183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51badda8-dc4a-4125-88ad-5dbc604b59f4 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:31.419995 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1363/2733] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:41:31 2022] PUT /identity/v3/projects/980fc587eb054036b5d7cde3f4d2a625/users/8cddba4b99e542499f27e0bf14f012b3/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:41:38.032087 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdb3a8-6d95-4424-be2b-9957d89f197b None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:38.032770 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdb3a8-6d95-4424-be2b-9957d89f197b None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:38.032770 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdb3a8-6d95-4424-be2b-9957d89f197b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:38.070019 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-25fdb3a8-6d95-4424-be2b-9957d89f197b None None] MFA Rules not processed for user `8cddba4b99e542499f27e0bf14f012b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:41:38.071534 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-25fdb3a8-6d95-4424-be2b-9957d89f197b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:38.173010 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1371/2734] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:41:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:41:38.188015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51badda8-dc4a-4125-88ad-5dbc604b59f4 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:38.190382 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-51badda8-dc4a-4125-88ad-5dbc604b59f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:38.258628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:41:38.327963 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51badda8-dc4a-4125-88ad-5dbc604b59f4 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:38.333926 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb88107-de61-4fae-af9f-63f1e2a77081 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:38.334628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb88107-de61-4fae-af9f-63f1e2a77081 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:38.334913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb88107-de61-4fae-af9f-63f1e2a77081 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:38.335191 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb88107-de61-4fae-af9f-63f1e2a77081 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:38.406787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb88107-de61-4fae-af9f-63f1e2a77081 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:38.411780 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb88107-de61-4fae-af9f-63f1e2a77081 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:38.424596 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1364/2735] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:41:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:41:48.171222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=81,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:41:48.641065 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=50 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:41:49.014032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6226ea8-282a-4c81-8e6a-398037d7c593 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:49.014350 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6226ea8-282a-4c81-8e6a-398037d7c593 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:49.014644 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6226ea8-282a-4c81-8e6a-398037d7c593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:49.042149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-f6226ea8-282a-4c81-8e6a-398037d7c593 None None] MFA Rules not processed for user `4e75a82cc7e94702ba0080fc65d6eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:41:49.043406 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f6226ea8-282a-4c81-8e6a-398037d7c593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:49.104009 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1372/2736] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Jul 20 15:41:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:41:49.288257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb88107-de61-4fae-af9f-63f1e2a77081 service nova] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:49.289673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-0eb88107-de61-4fae-af9f-63f1e2a77081 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:49.369017 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb88107-de61-4fae-af9f-63f1e2a77081 service nova] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:49.374820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bd401-0502-4b1b-8a71-e6115555b8cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0d719c0eb04a3b8a228c6ce70fdf98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:49.375540 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bd401-0502-4b1b-8a71-e6115555b8cf service swift] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:49.378423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bd401-0502-4b1b-8a71-e6115555b8cf service swift] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:49.378423 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bd401-0502-4b1b-8a71-e6115555b8cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:49.386030 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3bd401-0502-4b1b-8a71-e6115555b8cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:49.387733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3bd401-0502-4b1b-8a71-e6115555b8cf service swift] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:49.397172 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1365/2737] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Jul 20 15:41:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:41:49.792062 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6226ea8-282a-4c81-8e6a-398037d7c593 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:49.794978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f6226ea8-282a-4c81-8e6a-398037d7c593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:49.889066 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6226ea8-282a-4c81-8e6a-398037d7c593 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:49.892925 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec9905-ee2c-429e-9658-960ea77313f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:49.893367 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec9905-ee2c-429e-9658-960ea77313f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:49.893642 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec9905-ee2c-429e-9658-960ea77313f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:49.893923 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec9905-ee2c-429e-9658-960ea77313f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:49.895875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-fbec9905-ee2c-429e-9658-960ea77313f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:49.910559 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec9905-ee2c-429e-9658-960ea77313f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:49.912046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec9905-ee2c-429e-9658-960ea77313f8 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:49.925939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1373/2738] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:41:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:41:50.132211 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bd401-0502-4b1b-8a71-e6115555b8cf service swift] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:41:50.150370 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bd401-0502-4b1b-8a71-e6115555b8cf service swift] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:41:50.154447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f436b-582c-4e9e-880b-bfb9ebad3505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:41:50.155369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f436b-582c-4e9e-880b-bfb9ebad3505 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:41:50.155656 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f436b-582c-4e9e-880b-bfb9ebad3505 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:41:50.156027 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f436b-582c-4e9e-880b-bfb9ebad3505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:41:50.158500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-794f436b-582c-4e9e-880b-bfb9ebad3505 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:41:50.174537 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f436b-582c-4e9e-880b-bfb9ebad3505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:41:50.175995 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f436b-582c-4e9e-880b-bfb9ebad3505 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:41:50.189883 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1366/2739] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:41:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:41:59.919553 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=55 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:00.184324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=38 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:02.264194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469900fe-f124-40fe-b299-c5bc44e8e633 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:02.265314 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469900fe-f124-40fe-b299-c5bc44e8e633 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:02.265837 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469900fe-f124-40fe-b299-c5bc44e8e633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:02.287194 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-469900fe-f124-40fe-b299-c5bc44e8e633 None None] MFA Rules not processed for user `cb4dc884b7eb4aa8a0adcdb46c5f9586`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:42:02.288469 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-469900fe-f124-40fe-b299-c5bc44e8e633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:02.296994 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1374/2740] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:42:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:42:03.469940 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d47077-64a8-4cfd-a049-bc98d9094e9f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:03.471483 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d47077-64a8-4cfd-a049-bc98d9094e9f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:03.471483 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d47077-64a8-4cfd-a049-bc98d9094e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:03.491631 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-43d47077-64a8-4cfd-a049-bc98d9094e9f None None] MFA Rules not processed for user `0235b8fe92554131ba30e1d18f0f2d86`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:42:03.496166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-43d47077-64a8-4cfd-a049-bc98d9094e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:03.594310 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1367/2741] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:42:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:42:03.972813 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469900fe-f124-40fe-b299-c5bc44e8e633 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:03.977165 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-469900fe-f124-40fe-b299-c5bc44e8e633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:04.057006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469900fe-f124-40fe-b299-c5bc44e8e633 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:04.061669 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65ad48-e502-467e-ad32-e7cd89fc847f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:04.062695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65ad48-e502-467e-ad32-e7cd89fc847f service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:04.063231 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65ad48-e502-467e-ad32-e7cd89fc847f service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:04.063725 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65ad48-e502-467e-ad32-e7cd89fc847f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:04.065928 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-dc65ad48-e502-467e-ad32-e7cd89fc847f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:04.080030 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65ad48-e502-467e-ad32-e7cd89fc847f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:04.086024 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65ad48-e502-467e-ad32-e7cd89fc847f service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:04.089566 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d47077-64a8-4cfd-a049-bc98d9094e9f None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:04.091132 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-43d47077-64a8-4cfd-a049-bc98d9094e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:04.105029 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1375/2742] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:42:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:42:04.214336 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d47077-64a8-4cfd-a049-bc98d9094e9f None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:04.222049 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0d932-e102-40a8-a220-554291a910aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2192547babde4449a46f9cb37ed64524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:04.222766 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0d932-e102-40a8-a220-554291a910aa service placement] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:04.223373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0d932-e102-40a8-a220-554291a910aa service placement] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:04.223373 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0d932-e102-40a8-a220-554291a910aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:04.233738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-61d0d932-e102-40a8-a220-554291a910aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:04.248062 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0d932-e102-40a8-a220-554291a910aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:04.249980 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0d932-e102-40a8-a220-554291a910aa service placement] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:04.265962 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1368/2743] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:42:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:42:09.847388 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65ad48-e502-467e-ad32-e7cd89fc847f service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:09.858385 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65ad48-e502-467e-ad32-e7cd89fc847f service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:09.863235 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ded03-df82-471c-b110-14e80e68c3a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:09.864005 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ded03-df82-471c-b110-14e80e68c3a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:09.864392 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ded03-df82-471c-b110-14e80e68c3a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:09.864772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ded03-df82-471c-b110-14e80e68c3a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:09.866935 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9e8ded03-df82-471c-b110-14e80e68c3a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:09.881228 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ded03-df82-471c-b110-14e80e68c3a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:09.884454 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ded03-df82-471c-b110-14e80e68c3a8 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:09.896584 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1376/2744] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:42:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:42:10.216024 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe8d4d-ed4b-4a1f-b555-b83f7e65d3ac None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:10.216524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe8d4d-ed4b-4a1f-b555-b83f7e65d3ac None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:10.216646 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe8d4d-ed4b-4a1f-b555-b83f7e65d3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:10.267127 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-dbfe8d4d-ed4b-4a1f-b555-b83f7e65d3ac None None] MFA Rules not processed for user `4e75a82cc7e94702ba0080fc65d6eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:42:10.268406 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-dbfe8d4d-ed4b-4a1f-b555-b83f7e65d3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:10.287042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1369/2745] 10.222.0.78 () {58 vars in 967 bytes} [Wed Jul 20 15:42:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:42:10.315083 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ded03-df82-471c-b110-14e80e68c3a8 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:10.325433 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ded03-df82-471c-b110-14e80e68c3a8 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:10.331102 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd468f-b500-473d-ba7a-ac3b41a4d025 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0d719c0eb04a3b8a228c6ce70fdf98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:10.331862 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd468f-b500-473d-ba7a-ac3b41a4d025 service swift] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:10.332218 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd468f-b500-473d-ba7a-ac3b41a4d025 service swift] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:10.332570 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd468f-b500-473d-ba7a-ac3b41a4d025 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:10.341386 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd468f-b500-473d-ba7a-ac3b41a4d025 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:10.343063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd468f-b500-473d-ba7a-ac3b41a4d025 service swift] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:10.352854 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1377/2746] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Jul 20 15:42:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:42:15.280726 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe8d4d-ed4b-4a1f-b555-b83f7e65d3ac None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:15.280726 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-dbfe8d4d-ed4b-4a1f-b555-b83f7e65d3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:15.314149 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=100 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:15.319908 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe8d4d-ed4b-4a1f-b555-b83f7e65d3ac None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:15.325538 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430001e7-7bf9-47ce-a324-2b03c74bee0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:15.327388 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430001e7-7bf9-47ce-a324-2b03c74bee0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:15.327877 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430001e7-7bf9-47ce-a324-2b03c74bee0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:15.328361 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430001e7-7bf9-47ce-a324-2b03c74bee0f admin admin] PATH_INFO: `/v3/users/a86c442d85e34a3487c8ad101045ee54` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:15.329786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430001e7-7bf9-47ce-a324-2b03c74bee0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a86c442d85e34a3487c8ad101045ee54)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:15.331973 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430001e7-7bf9-47ce-a324-2b03c74bee0f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:15.735812 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-430001e7-7bf9-47ce-a324-2b03c74bee0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a86c442d85e34a3487c8ad101045ee54'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:15.914796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-430001e7-7bf9-47ce-a324-2b03c74bee0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a86c442d85e34a3487c8ad101045ee54'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:15.915430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-430001e7-7bf9-47ce-a324-2b03c74bee0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a86c442d85e34a3487c8ad101045ee54'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:15.928614 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-430001e7-7bf9-47ce-a324-2b03c74bee0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a86c442d85e34a3487c8ad101045ee54'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:15.935401 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-430001e7-7bf9-47ce-a324-2b03c74bee0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a86c442d85e34a3487c8ad101045ee54'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:15.942720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1370/2747] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:42:15 2022] DELETE /identity/v3/users/a86c442d85e34a3487c8ad101045ee54 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:42:16.267728 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd468f-b500-473d-ba7a-ac3b41a4d025 service swift] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:16.281575 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd468f-b500-473d-ba7a-ac3b41a4d025 service swift] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:16.355383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7df3a-0129-4186-9b30-734202b4cc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:16.356310 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7df3a-0129-4186-9b30-734202b4cc47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:16.356539 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7df3a-0129-4186-9b30-734202b4cc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:16.356841 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7df3a-0129-4186-9b30-734202b4cc47 admin admin] PATH_INFO: `/v3/projects/093f88479eae442dba4d8320fbcb2b40` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:16.358186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7df3a-0129-4186-9b30-734202b4cc47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=093f88479eae442dba4d8320fbcb2b40)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:16.360226 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7df3a-0129-4186-9b30-734202b4cc47 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:16.596287 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-cfa7df3a-0129-4186-9b30-734202b4cc47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '093f88479eae442dba4d8320fbcb2b40', 'request_id': 'req-cfa7df3a-0129-4186-9b30-734202b4cc47'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:16.771451 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-cfa7df3a-0129-4186-9b30-734202b4cc47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '093f88479eae442dba4d8320fbcb2b40', 'request_id': 'req-cfa7df3a-0129-4186-9b30-734202b4cc47'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:16.781780 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-cfa7df3a-0129-4186-9b30-734202b4cc47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '093f88479eae442dba4d8320fbcb2b40', 'request_id': 'req-cfa7df3a-0129-4186-9b30-734202b4cc47'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:16.785355 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:16.796032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-cfa7df3a-0129-4186-9b30-734202b4cc47 admin admin] The token cache is being invalidate because project 093f88479eae442dba4d8320fbcb2b40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:42:16.796331 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-cfa7df3a-0129-4186-9b30-734202b4cc47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:16.796997 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-cfa7df3a-0129-4186-9b30-734202b4cc47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:16.798595 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1378/2748] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:42:16 2022] DELETE /identity/v3/projects/093f88479eae442dba4d8320fbcb2b40 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:42:16.804123 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f842ad0-cf33-4869-a857-7c2ee2139e8a None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:16.804674 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f842ad0-cf33-4869-a857-7c2ee2139e8a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:16.804779 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f842ad0-cf33-4869-a857-7c2ee2139e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:16.824701 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-1f842ad0-cf33-4869-a857-7c2ee2139e8a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:42:16.826242 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-1f842ad0-cf33-4869-a857-7c2ee2139e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:16.912072 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1371/2749] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:42:16.922521 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07d998-6121-4ad5-8085-83c6c6e61071 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:16.922803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07d998-6121-4ad5-8085-83c6c6e61071 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:16.923086 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07d998-6121-4ad5-8085-83c6c6e61071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:16.949401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-dc07d998-6121-4ad5-8085-83c6c6e61071 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:42:16.951755 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-dc07d998-6121-4ad5-8085-83c6c6e61071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:16.964491 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1379/2750] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:42:16.965490 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f842ad0-cf33-4869-a857-7c2ee2139e8a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:16.975757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f842ad0-cf33-4869-a857-7c2ee2139e8a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:16.987805 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a92273-87f3-4901-82ac-a83463c6dc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:16.988502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a92273-87f3-4901-82ac-a83463c6dc46 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:16.988787 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a92273-87f3-4901-82ac-a83463c6dc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:16.989093 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a92273-87f3-4901-82ac-a83463c6dc46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:16.989866 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a92273-87f3-4901-82ac-a83463c6dc46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:42:16.990180 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a92273-87f3-4901-82ac-a83463c6dc46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:16.991824 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a92273-87f3-4901-82ac-a83463c6dc46 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:17.004372 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1372/2751] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:42:17.008761 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07d998-6121-4ad5-8085-83c6c6e61071 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:17.018743 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07d998-6121-4ad5-8085-83c6c6e61071 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:17.024861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fbaf4-3c69-41dc-af77-b4ff40495427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:17.026013 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fbaf4-3c69-41dc-af77-b4ff40495427 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:17.026332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fbaf4-3c69-41dc-af77-b4ff40495427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:17.026606 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fbaf4-3c69-41dc-af77-b4ff40495427 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:17.028094 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4fbaf4-3c69-41dc-af77-b4ff40495427 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:17.030190 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4fbaf4-3c69-41dc-af77-b4ff40495427 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:17.123006 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1380/2752] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:42:17 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 15:42:17.127227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a92273-87f3-4901-82ac-a83463c6dc46 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:17.137785 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a92273-87f3-4901-82ac-a83463c6dc46 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:17.204548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ebca9-0aaa-4cac-a4f2-f6a96e0ac4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:17.205224 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ebca9-0aaa-4cac-a4f2-f6a96e0ac4b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:17.205515 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ebca9-0aaa-4cac-a4f2-f6a96e0ac4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:17.205796 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ebca9-0aaa-4cac-a4f2-f6a96e0ac4b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:17.206547 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ebca9-0aaa-4cac-a4f2-f6a96e0ac4b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:17.208449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ebca9-0aaa-4cac-a4f2-f6a96e0ac4b8 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:17.217060 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:17.287251 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1373/2753] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:42:17 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:42:17.291738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fbaf4-3c69-41dc-af77-b4ff40495427 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:17.302967 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fbaf4-3c69-41dc-af77-b4ff40495427 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:17.308196 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a07ba-525b-45b0-a573-1b1e218c0633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:17.308874 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a07ba-525b-45b0-a573-1b1e218c0633 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:17.309160 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a07ba-525b-45b0-a573-1b1e218c0633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:17.309464 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a07ba-525b-45b0-a573-1b1e218c0633 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:17.310195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a07ba-525b-45b0-a573-1b1e218c0633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:17.311882 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a07ba-525b-45b0-a573-1b1e218c0633 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:17.322518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1381/2754] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:42:17.326977 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ebca9-0aaa-4cac-a4f2-f6a96e0ac4b8 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:17.338580 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ebca9-0aaa-4cac-a4f2-f6a96e0ac4b8 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:17.343300 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0b1f5-dec8-494f-8258-2a54bba5ff75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:17.343961 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0b1f5-dec8-494f-8258-2a54bba5ff75 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:17.344245 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0b1f5-dec8-494f-8258-2a54bba5ff75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:17.344524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0b1f5-dec8-494f-8258-2a54bba5ff75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:17.345251 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0b1f5-dec8-494f-8258-2a54bba5ff75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:17.346853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0b1f5-dec8-494f-8258-2a54bba5ff75 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:17.358557 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1374/2755] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:42:17.362901 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a07ba-525b-45b0-a573-1b1e218c0633 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:17.373459 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a07ba-525b-45b0-a573-1b1e218c0633 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:17.378329 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd749d5-4704-4d01-a8d2-a1bde077d054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:17.379222 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd749d5-4704-4d01-a8d2-a1bde077d054 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:17.379504 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd749d5-4704-4d01-a8d2-a1bde077d054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:17.379780 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd749d5-4704-4d01-a8d2-a1bde077d054 admin admin] PATH_INFO: `/v3/projects/e8cb9fef28e1414a8febf9382281b65e/users/f32138b86095453db243ea0ca97b1404/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:17.395502 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd749d5-4704-4d01-a8d2-a1bde077d054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8cb9fef28e1414a8febf9382281b65e, user_id=f32138b86095453db243ea0ca97b1404, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:17.398144 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd749d5-4704-4d01-a8d2-a1bde077d054 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:17.474896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1382/2756] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:42:17 2022] PUT /identity/v3/projects/e8cb9fef28e1414a8febf9382281b65e/users/f32138b86095453db243ea0ca97b1404/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:42:17.486424 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977785d-18d8-4450-af44-2649fe61cfd9 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:17.486717 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977785d-18d8-4450-af44-2649fe61cfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:17.487019 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977785d-18d8-4450-af44-2649fe61cfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:17.514412 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-b977785d-18d8-4450-af44-2649fe61cfd9 None None] MFA Rules not processed for user `f32138b86095453db243ea0ca97b1404`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:42:17.515871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b977785d-18d8-4450-af44-2649fe61cfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:17.581310 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1375/2757] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:42:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:42:17.592934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd749d5-4704-4d01-a8d2-a1bde077d054 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:17.594734 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0fd749d5-4704-4d01-a8d2-a1bde077d054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:17.652808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd749d5-4704-4d01-a8d2-a1bde077d054 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:17.656578 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5e5e7-5c2f-41b2-924f-0de7231f2bba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:17.657231 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5e5e7-5c2f-41b2-924f-0de7231f2bba service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:17.657517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5e5e7-5c2f-41b2-924f-0de7231f2bba service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:17.657796 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5e5e7-5c2f-41b2-924f-0de7231f2bba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:17.666321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5e5e7-5c2f-41b2-924f-0de7231f2bba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:17.667793 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5e5e7-5c2f-41b2-924f-0de7231f2bba service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:17.676539 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1383/2758] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:42:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:42:17.701820 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977785d-18d8-4450-af44-2649fe61cfd9 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:17.772109 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977785d-18d8-4450-af44-2649fe61cfd9 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:17.778554 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e4638-2d77-4b2e-a58d-94989e4f9b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:17.779467 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e4638-2d77-4b2e-a58d-94989e4f9b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:17.779750 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e4638-2d77-4b2e-a58d-94989e4f9b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:17.780021 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e4638-2d77-4b2e-a58d-94989e4f9b03 admin admin] PATH_INFO: `/v3/users/f32138b86095453db243ea0ca97b1404` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:17.781333 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e4638-2d77-4b2e-a58d-94989e4f9b03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f32138b86095453db243ea0ca97b1404)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:17.783532 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e4638-2d77-4b2e-a58d-94989e4f9b03 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:17.963801 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-1c5e4638-2d77-4b2e-a58d-94989e4f9b03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f32138b86095453db243ea0ca97b1404'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:18.046563 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-1c5e4638-2d77-4b2e-a58d-94989e4f9b03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f32138b86095453db243ea0ca97b1404'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:18.047625 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-1c5e4638-2d77-4b2e-a58d-94989e4f9b03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f32138b86095453db243ea0ca97b1404'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:18.061698 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-1c5e4638-2d77-4b2e-a58d-94989e4f9b03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f32138b86095453db243ea0ca97b1404'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:18.062947 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:18.069350 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-1c5e4638-2d77-4b2e-a58d-94989e4f9b03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f32138b86095453db243ea0ca97b1404'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:18.077018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1376/2759] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:42:17 2022] DELETE /identity/v3/users/f32138b86095453db243ea0ca97b1404 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:42:18.081562 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0501d-60b3-450e-85e6-8cb89e86215f None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:18.081852 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0501d-60b3-450e-85e6-8cb89e86215f None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:18.082131 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0501d-60b3-450e-85e6-8cb89e86215f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:18.100828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-dfa0501d-60b3-450e-85e6-8cb89e86215f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:42:18.102069 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-dfa0501d-60b3-450e-85e6-8cb89e86215f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:18.163315 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1384/2760] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:42:18.172724 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e4638-2d77-4b2e-a58d-94989e4f9b03 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:18.174156 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-1c5e4638-2d77-4b2e-a58d-94989e4f9b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:18.233637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e4638-2d77-4b2e-a58d-94989e4f9b03 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:18.237235 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2333e19-6579-4456-a69a-0cfae5217ee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:18.237884 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2333e19-6579-4456-a69a-0cfae5217ee3 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:18.238178 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2333e19-6579-4456-a69a-0cfae5217ee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:18.238691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2333e19-6579-4456-a69a-0cfae5217ee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:18.248586 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2333e19-6579-4456-a69a-0cfae5217ee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:18.250003 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2333e19-6579-4456-a69a-0cfae5217ee3 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:18.259320 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1377/2761] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:42:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:42:18.665684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0501d-60b3-450e-85e6-8cb89e86215f None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:18.667917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-dfa0501d-60b3-450e-85e6-8cb89e86215f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:18.680709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0501d-60b3-450e-85e6-8cb89e86215f None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:18.687393 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e24aa-4deb-4439-a70d-c5b5a44f724b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:18.688276 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e24aa-4deb-4439-a70d-c5b5a44f724b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:18.688620 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e24aa-4deb-4439-a70d-c5b5a44f724b admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:18.688869 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e24aa-4deb-4439-a70d-c5b5a44f724b admin admin] PATH_INFO: `/v3/projects/e8cb9fef28e1414a8febf9382281b65e` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:18.690094 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e24aa-4deb-4439-a70d-c5b5a44f724b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8cb9fef28e1414a8febf9382281b65e)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:18.692040 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e24aa-4deb-4439-a70d-c5b5a44f724b admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:18.709440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:18.819462 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ca8e24aa-4deb-4439-a70d-c5b5a44f724b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8cb9fef28e1414a8febf9382281b65e', 'request_id': 'req-ca8e24aa-4deb-4439-a70d-c5b5a44f724b'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:18.903677 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ca8e24aa-4deb-4439-a70d-c5b5a44f724b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8cb9fef28e1414a8febf9382281b65e', 'request_id': 'req-ca8e24aa-4deb-4439-a70d-c5b5a44f724b'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:18.910509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ca8e24aa-4deb-4439-a70d-c5b5a44f724b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8cb9fef28e1414a8febf9382281b65e', 'request_id': 'req-ca8e24aa-4deb-4439-a70d-c5b5a44f724b'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:18.922229 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ca8e24aa-4deb-4439-a70d-c5b5a44f724b admin admin] The token cache is being invalidate because project e8cb9fef28e1414a8febf9382281b65e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:42:18.922547 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ca8e24aa-4deb-4439-a70d-c5b5a44f724b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:18.923205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-ca8e24aa-4deb-4439-a70d-c5b5a44f724b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:18.924760 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1385/2762] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:42:18 2022] DELETE /identity/v3/projects/e8cb9fef28e1414a8febf9382281b65e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:42:18.929605 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ef927-b7e8-494c-a1cf-66dc236d176a None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:18.929900 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ef927-b7e8-494c-a1cf-66dc236d176a None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:18.930231 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ef927-b7e8-494c-a1cf-66dc236d176a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:18.948789 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-5c0ef927-b7e8-494c-a1cf-66dc236d176a None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:42:18.950052 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5c0ef927-b7e8-494c-a1cf-66dc236d176a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:19.023829 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1378/2763] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:42:19.032756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb16abc-9dbf-401f-b651-d607566d3f85 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:19.033050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb16abc-9dbf-401f-b651-d607566d3f85 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:19.033331 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb16abc-9dbf-401f-b651-d607566d3f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:19.054010 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-5eb16abc-9dbf-401f-b651-d607566d3f85 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:42:19.055269 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-5eb16abc-9dbf-401f-b651-d607566d3f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:19.063543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1386/2764] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:42:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:42:19.068163 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ef927-b7e8-494c-a1cf-66dc236d176a None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:19.078387 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ef927-b7e8-494c-a1cf-66dc236d176a None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:19.083079 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bb7e9-b25c-49cc-bdfd-92326049fe26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:19.083747 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bb7e9-b25c-49cc-bdfd-92326049fe26 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:19.084034 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bb7e9-b25c-49cc-bdfd-92326049fe26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:19.084316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bb7e9-b25c-49cc-bdfd-92326049fe26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:19.085081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bb7e9-b25c-49cc-bdfd-92326049fe26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:42:19.085398 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bb7e9-b25c-49cc-bdfd-92326049fe26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:19.087039 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bb7e9-b25c-49cc-bdfd-92326049fe26 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:19.100656 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1379/2765] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:42:19.105557 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb16abc-9dbf-401f-b651-d607566d3f85 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:19.115008 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb16abc-9dbf-401f-b651-d607566d3f85 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:19.120839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e936699-adb0-471e-a896-d342e9c0a1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:19.121496 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e936699-adb0-471e-a896-d342e9c0a1a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:19.121775 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e936699-adb0-471e-a896-d342e9c0a1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:19.122048 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e936699-adb0-471e-a896-d342e9c0a1a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:19.122764 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e936699-adb0-471e-a896-d342e9c0a1a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:19.124655 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e936699-adb0-471e-a896-d342e9c0a1a7 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:19.249348 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1387/2766] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:42:19 2022] POST /identity/v3/projects => generated 391 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:42:19.250625 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bb7e9-b25c-49cc-bdfd-92326049fe26 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:19.263195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bb7e9-b25c-49cc-bdfd-92326049fe26 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:19.334826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4471a-0697-495c-a4f3-41cce51e3d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:19.335516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4471a-0697-495c-a4f3-41cce51e3d8b admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:19.335808 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4471a-0697-495c-a4f3-41cce51e3d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:19.336105 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4471a-0697-495c-a4f3-41cce51e3d8b admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:19.336973 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4471a-0697-495c-a4f3-41cce51e3d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:19.340188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4471a-0697-495c-a4f3-41cce51e3d8b admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:19.427083 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1380/2767] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:42:19 2022] POST /identity/v3/users => generated 296 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:42:19.428259 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e936699-adb0-471e-a896-d342e9c0a1a7 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:19.438838 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e936699-adb0-471e-a896-d342e9c0a1a7 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:19.443761 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea16590-c06e-4c8c-98ba-1726cdb5137b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:19.444442 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea16590-c06e-4c8c-98ba-1726cdb5137b admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:19.444726 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea16590-c06e-4c8c-98ba-1726cdb5137b admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:19.445000 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea16590-c06e-4c8c-98ba-1726cdb5137b admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:19.445719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea16590-c06e-4c8c-98ba-1726cdb5137b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:19.447887 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea16590-c06e-4c8c-98ba-1726cdb5137b admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:19.459137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1388/2768] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:42:19.463954 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4471a-0697-495c-a4f3-41cce51e3d8b admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:19.474621 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4471a-0697-495c-a4f3-41cce51e3d8b admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:19.479360 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e58298-7f3d-490f-996f-5647e0947bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:19.480055 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e58298-7f3d-490f-996f-5647e0947bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:19.480341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e58298-7f3d-490f-996f-5647e0947bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:19.480627 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e58298-7f3d-490f-996f-5647e0947bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:19.481359 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e58298-7f3d-490f-996f-5647e0947bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:19.482995 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e58298-7f3d-490f-996f-5647e0947bf6 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:19.485480 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:19.494548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1381/2769] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:42:19.500602 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea16590-c06e-4c8c-98ba-1726cdb5137b admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:19.511465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea16590-c06e-4c8c-98ba-1726cdb5137b admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:19.516312 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2bdae-549d-4b4f-8d0d-441d6f1be0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:19.517208 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2bdae-549d-4b4f-8d0d-441d6f1be0e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:19.517520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2bdae-549d-4b4f-8d0d-441d6f1be0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:19.517803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2bdae-549d-4b4f-8d0d-441d6f1be0e2 admin admin] PATH_INFO: `/v3/projects/044407e74d224d3badd4c695c51601c5/users/bef14afe93ae402aa2a611c6edaaea61/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:19.540600 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2bdae-549d-4b4f-8d0d-441d6f1be0e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=044407e74d224d3badd4c695c51601c5, user_id=bef14afe93ae402aa2a611c6edaaea61, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:19.543661 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2bdae-549d-4b4f-8d0d-441d6f1be0e2 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:19.645648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1389/2770] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:42:19 2022] PUT /identity/v3/projects/044407e74d224d3badd4c695c51601c5/users/bef14afe93ae402aa2a611c6edaaea61/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:42:19.656182 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88333e-67e8-4992-acab-7301d271af3f None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:19.656476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88333e-67e8-4992-acab-7301d271af3f None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:19.656756 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88333e-67e8-4992-acab-7301d271af3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:19.685575 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-8d88333e-67e8-4992-acab-7301d271af3f None None] MFA Rules not processed for user `bef14afe93ae402aa2a611c6edaaea61`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:42:19.686838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8d88333e-67e8-4992-acab-7301d271af3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:19.751737 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1382/2771] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:42:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:42:19.762923 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2bdae-549d-4b4f-8d0d-441d6f1be0e2 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:19.765053 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-cbe2bdae-549d-4b4f-8d0d-441d6f1be0e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:19.828743 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2bdae-549d-4b4f-8d0d-441d6f1be0e2 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:19.832447 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b5c5d-de4a-4cbe-9a1e-ae9352f5b2ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:19.833214 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b5c5d-de4a-4cbe-9a1e-ae9352f5b2ca service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:19.833507 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b5c5d-de4a-4cbe-9a1e-ae9352f5b2ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:19.833785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b5c5d-de4a-4cbe-9a1e-ae9352f5b2ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:19.843671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b5c5d-de4a-4cbe-9a1e-ae9352f5b2ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:19.845101 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b5c5d-de4a-4cbe-9a1e-ae9352f5b2ca service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:19.855142 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1390/2772] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:42:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:42:29.748649 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=33 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:29.850613 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:32.533428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88333e-67e8-4992-acab-7301d271af3f None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:32.534862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8d88333e-67e8-4992-acab-7301d271af3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:32.592073 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88333e-67e8-4992-acab-7301d271af3f None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:32.596273 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86e076-e902-4727-bec8-9afbe074b34e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:32.596933 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86e076-e902-4727-bec8-9afbe074b34e service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:32.597219 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86e076-e902-4727-bec8-9afbe074b34e service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:32.597499 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86e076-e902-4727-bec8-9afbe074b34e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:32.599426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-3e86e076-e902-4727-bec8-9afbe074b34e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:32.661023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86e076-e902-4727-bec8-9afbe074b34e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:32.662470 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86e076-e902-4727-bec8-9afbe074b34e service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:32.671859 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1383/2773] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:42:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:42:35.191726 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b5c5d-de4a-4cbe-9a1e-ae9352f5b2ca service cinder] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:35.193155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-c52b5c5d-de4a-4cbe-9a1e-ae9352f5b2ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:35.271941 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b5c5d-de4a-4cbe-9a1e-ae9352f5b2ca service cinder] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:35.278555 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46750c-e2e1-4815-9594-cd75db3452b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:35.279224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46750c-e2e1-4815-9594-cd75db3452b1 service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:35.279510 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46750c-e2e1-4815-9594-cd75db3452b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:35.279789 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46750c-e2e1-4815-9594-cd75db3452b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:35.281639 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0d46750c-e2e1-4815-9594-cd75db3452b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:35.295767 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46750c-e2e1-4815-9594-cd75db3452b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:35.297372 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46750c-e2e1-4815-9594-cd75db3452b1 service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:35.309619 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1391/2774] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:42:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:42:41.735883 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86e076-e902-4727-bec8-9afbe074b34e service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:41.799132 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86e076-e902-4727-bec8-9afbe074b34e service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:41.803838 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fe12d-315f-489f-ab53-e4c244573f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:41.806687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fe12d-315f-489f-ab53-e4c244573f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:41.806687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fe12d-315f-489f-ab53-e4c244573f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:41.806687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fe12d-315f-489f-ab53-e4c244573f69 admin admin] PATH_INFO: `/v3/users/bef14afe93ae402aa2a611c6edaaea61` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:41.806687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fe12d-315f-489f-ab53-e4c244573f69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bef14afe93ae402aa2a611c6edaaea61)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:41.808724 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fe12d-315f-489f-ab53-e4c244573f69 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:42.075963 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-c14fe12d-315f-489f-ab53-e4c244573f69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bef14afe93ae402aa2a611c6edaaea61'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:42.079616 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:42.159200 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-c14fe12d-315f-489f-ab53-e4c244573f69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bef14afe93ae402aa2a611c6edaaea61'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:42.159876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-c14fe12d-315f-489f-ab53-e4c244573f69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bef14afe93ae402aa2a611c6edaaea61'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:42.172630 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-c14fe12d-315f-489f-ab53-e4c244573f69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bef14afe93ae402aa2a611c6edaaea61'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:42.179428 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-c14fe12d-315f-489f-ab53-e4c244573f69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bef14afe93ae402aa2a611c6edaaea61'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:42.185842 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1384/2775] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:42:41 2022] DELETE /identity/v3/users/bef14afe93ae402aa2a611c6edaaea61 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:42:42.190411 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447daf18-410d-4bcd-945a-4c6e212c1e8f None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:42.190934 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447daf18-410d-4bcd-945a-4c6e212c1e8f None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:42.191063 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447daf18-410d-4bcd-945a-4c6e212c1e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:42.210123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-447daf18-410d-4bcd-945a-4c6e212c1e8f None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:42:42.211354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-447daf18-410d-4bcd-945a-4c6e212c1e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:42.269949 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1392/2776] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:42:42.280564 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fe12d-315f-489f-ab53-e4c244573f69 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:42.285740 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-c14fe12d-315f-489f-ab53-e4c244573f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:42.338909 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fe12d-315f-489f-ab53-e4c244573f69 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:42.342487 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a0725-2a47-444d-abd2-1f85f41a5430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:42.343125 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a0725-2a47-444d-abd2-1f85f41a5430 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:42.343406 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a0725-2a47-444d-abd2-1f85f41a5430 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:42.343681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a0725-2a47-444d-abd2-1f85f41a5430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:42.352461 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a0725-2a47-444d-abd2-1f85f41a5430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:42.353871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a0725-2a47-444d-abd2-1f85f41a5430 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:42.365774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1385/2777] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:42:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:42:42.974469 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447daf18-410d-4bcd-945a-4c6e212c1e8f None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:42:42.974469 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-447daf18-410d-4bcd-945a-4c6e212c1e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:42:42.986025 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447daf18-410d-4bcd-945a-4c6e212c1e8f None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:42:42.990650 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100ee19-15d8-4984-aac4-535135782a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:42:42.991517 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100ee19-15d8-4984-aac4-535135782a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:42:42.991788 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100ee19-15d8-4984-aac4-535135782a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:42:42.992059 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100ee19-15d8-4984-aac4-535135782a75 admin admin] PATH_INFO: `/v3/projects/044407e74d224d3badd4c695c51601c5` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:42:42.994441 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100ee19-15d8-4984-aac4-535135782a75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=044407e74d224d3badd4c695c51601c5)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:42:42.996423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100ee19-15d8-4984-aac4-535135782a75 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:42:43.179649 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-9100ee19-15d8-4984-aac4-535135782a75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '044407e74d224d3badd4c695c51601c5', 'request_id': 'req-9100ee19-15d8-4984-aac4-535135782a75'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:43.400347 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-9100ee19-15d8-4984-aac4-535135782a75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '044407e74d224d3badd4c695c51601c5', 'request_id': 'req-9100ee19-15d8-4984-aac4-535135782a75'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:43.408323 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-9100ee19-15d8-4984-aac4-535135782a75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '044407e74d224d3badd4c695c51601c5', 'request_id': 'req-9100ee19-15d8-4984-aac4-535135782a75'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:43.420316 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-9100ee19-15d8-4984-aac4-535135782a75 admin admin] The token cache is being invalidate because project 044407e74d224d3badd4c695c51601c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:42:43.420604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-9100ee19-15d8-4984-aac4-535135782a75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:43.421270 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-9100ee19-15d8-4984-aac4-535135782a75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:42:43.422965 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1393/2778] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:42:42 2022] DELETE /identity/v3/projects/044407e74d224d3badd4c695c51601c5 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:42:52.359788 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=38,keystone:DELETE=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:42:53.417090 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:04.818234 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a0725-2a47-444d-abd2-1f85f41a5430 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:04.819708 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-259a0725-2a47-444d-abd2-1f85f41a5430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:04.879068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a0725-2a47-444d-abd2-1f85f41a5430 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:04.930530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78959110-e7fa-4230-ab5e-ae78954e7b83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:04.931225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78959110-e7fa-4230-ab5e-ae78954e7b83 service cinder] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:04.931524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78959110-e7fa-4230-ab5e-ae78954e7b83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:04.931807 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78959110-e7fa-4230-ab5e-ae78954e7b83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:04.933704 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-78959110-e7fa-4230-ab5e-ae78954e7b83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:04.992741 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78959110-e7fa-4230-ab5e-ae78954e7b83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:04.994171 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78959110-e7fa-4230-ab5e-ae78954e7b83 service cinder] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:05.003158 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1386/2779] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:43:13.305262 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100ee19-15d8-4984-aac4-535135782a75 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:13.307361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-9100ee19-15d8-4984-aac4-535135782a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:13.394988 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100ee19-15d8-4984-aac4-535135782a75 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:13.405022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d1717-9f4f-4d42-be6d-ceec5a251711 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:13.405022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d1717-9f4f-4d42-be6d-ceec5a251711 service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:13.405022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d1717-9f4f-4d42-be6d-ceec5a251711 service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:13.405022 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d1717-9f4f-4d42-be6d-ceec5a251711 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:13.405549 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2b8d1717-9f4f-4d42-be6d-ceec5a251711 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:13.419195 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d1717-9f4f-4d42-be6d-ceec5a251711 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:13.421209 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d1717-9f4f-4d42-be6d-ceec5a251711 service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:13.438970 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1394/2780] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:43:14.998890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=68 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:20.409765 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78959110-e7fa-4230-ab5e-ae78954e7b83 service cinder] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:20.413242 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-78959110-e7fa-4230-ab5e-ae78954e7b83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:20.471718 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78959110-e7fa-4230-ab5e-ae78954e7b83 service cinder] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:20.475481 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759b82a-fda0-417a-9500-323b3bab3da0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:20.482588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759b82a-fda0-417a-9500-323b3bab3da0 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:20.482588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759b82a-fda0-417a-9500-323b3bab3da0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:20.482588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759b82a-fda0-417a-9500-323b3bab3da0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:20.482588 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-8759b82a-fda0-417a-9500-323b3bab3da0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:20.550432 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759b82a-fda0-417a-9500-323b3bab3da0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:20.551775 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759b82a-fda0-417a-9500-323b3bab3da0 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:20.562309 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1387/2781] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:43:23.432021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=34 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:24.631104 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d1717-9f4f-4d42-be6d-ceec5a251711 service nova] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:24.633645 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-2b8d1717-9f4f-4d42-be6d-ceec5a251711 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:24.647626 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d1717-9f4f-4d42-be6d-ceec5a251711 service nova] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:24.652203 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331097ad-6df1-4d3b-99d7-37b4c2736c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:24.653084 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331097ad-6df1-4d3b-99d7-37b4c2736c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:24.653368 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331097ad-6df1-4d3b-99d7-37b4c2736c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:24.653834 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331097ad-6df1-4d3b-99d7-37b4c2736c3d admin admin] PATH_INFO: `/v3/users/0235b8fe92554131ba30e1d18f0f2d86` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:24.655070 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331097ad-6df1-4d3b-99d7-37b4c2736c3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0235b8fe92554131ba30e1d18f0f2d86)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:24.656976 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331097ad-6df1-4d3b-99d7-37b4c2736c3d admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:25.028685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-331097ad-6df1-4d3b-99d7-37b4c2736c3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0235b8fe92554131ba30e1d18f0f2d86'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:25.197738 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-331097ad-6df1-4d3b-99d7-37b4c2736c3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0235b8fe92554131ba30e1d18f0f2d86'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:25.205963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-331097ad-6df1-4d3b-99d7-37b4c2736c3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0235b8fe92554131ba30e1d18f0f2d86'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:25.207905 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-331097ad-6df1-4d3b-99d7-37b4c2736c3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0235b8fe92554131ba30e1d18f0f2d86'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:25.219324 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-331097ad-6df1-4d3b-99d7-37b4c2736c3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0235b8fe92554131ba30e1d18f0f2d86'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:25.225480 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1395/2782] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:43:24 2022] DELETE /identity/v3/users/0235b8fe92554131ba30e1d18f0f2d86 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:25.553303 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759b82a-fda0-417a-9500-323b3bab3da0 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:25.567299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759b82a-fda0-417a-9500-323b3bab3da0 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:25.628435 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66ee21-d045-4133-a829-284b28df0831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:25.629219 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66ee21-d045-4133-a829-284b28df0831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:25.629503 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66ee21-d045-4133-a829-284b28df0831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:25.629778 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66ee21-d045-4133-a829-284b28df0831 admin admin] PATH_INFO: `/v3/projects/44dc2a5c754d42d1862b331afef2183f` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:25.630977 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66ee21-d045-4133-a829-284b28df0831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44dc2a5c754d42d1862b331afef2183f)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:25.632865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66ee21-d045-4133-a829-284b28df0831 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:25.748894 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ff66ee21-d045-4133-a829-284b28df0831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44dc2a5c754d42d1862b331afef2183f', 'request_id': 'req-ff66ee21-d045-4133-a829-284b28df0831'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:25.751454 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:25.879377 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ff66ee21-d045-4133-a829-284b28df0831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44dc2a5c754d42d1862b331afef2183f', 'request_id': 'req-ff66ee21-d045-4133-a829-284b28df0831'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:25.886293 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ff66ee21-d045-4133-a829-284b28df0831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44dc2a5c754d42d1862b331afef2183f', 'request_id': 'req-ff66ee21-d045-4133-a829-284b28df0831'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:25.897691 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ff66ee21-d045-4133-a829-284b28df0831 admin admin] The token cache is being invalidate because project 44dc2a5c754d42d1862b331afef2183f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:43:25.897976 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ff66ee21-d045-4133-a829-284b28df0831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:25.898641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ff66ee21-d045-4133-a829-284b28df0831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:25.900244 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1388/2783] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:43:25 2022] DELETE /identity/v3/projects/44dc2a5c754d42d1862b331afef2183f => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:25.905207 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb93b7b-243e-42bd-b7a8-d7e872268e44 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:25.905685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb93b7b-243e-42bd-b7a8-d7e872268e44 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:25.905818 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb93b7b-243e-42bd-b7a8-d7e872268e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:25.924920 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-dfb93b7b-243e-42bd-b7a8-d7e872268e44 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:25.926161 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-dfb93b7b-243e-42bd-b7a8-d7e872268e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:25.986746 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1396/2784] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:25.994902 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeab032-c809-4672-b9d6-57a67511a8f9 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:25.995186 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeab032-c809-4672-b9d6-57a67511a8f9 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:25.998197 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeab032-c809-4672-b9d6-57a67511a8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:26.013508 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-feeab032-c809-4672-b9d6-57a67511a8f9 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:26.014829 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-feeab032-c809-4672-b9d6-57a67511a8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:26.022660 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1389/2785] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:26.026995 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb93b7b-243e-42bd-b7a8-d7e872268e44 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:26.036817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb93b7b-243e-42bd-b7a8-d7e872268e44 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:26.042027 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5302e-78d5-4656-ae3f-405c91f2f75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:26.042730 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5302e-78d5-4656-ae3f-405c91f2f75e admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:26.043018 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5302e-78d5-4656-ae3f-405c91f2f75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:26.043301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5302e-78d5-4656-ae3f-405c91f2f75e admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:26.044031 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5302e-78d5-4656-ae3f-405c91f2f75e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:43:26.044336 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5302e-78d5-4656-ae3f-405c91f2f75e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:26.045921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5302e-78d5-4656-ae3f-405c91f2f75e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:26.058696 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1397/2786] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:43:26.063081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeab032-c809-4672-b9d6-57a67511a8f9 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:26.073245 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeab032-c809-4672-b9d6-57a67511a8f9 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:26.078200 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc910a7-f534-4484-94fe-14a6b9cdcfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:26.078886 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc910a7-f534-4484-94fe-14a6b9cdcfba admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:26.079175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc910a7-f534-4484-94fe-14a6b9cdcfba admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:26.079453 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc910a7-f534-4484-94fe-14a6b9cdcfba admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:26.080157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc910a7-f534-4484-94fe-14a6b9cdcfba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:26.082057 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc910a7-f534-4484-94fe-14a6b9cdcfba admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:26.193928 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1390/2787] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:43:26 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:43:26.198135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5302e-78d5-4656-ae3f-405c91f2f75e admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:26.208501 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5302e-78d5-4656-ae3f-405c91f2f75e admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:26.265509 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e38a8d-49fa-43e8-afee-c897572b6405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:26.266209 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e38a8d-49fa-43e8-afee-c897572b6405 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:26.266530 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e38a8d-49fa-43e8-afee-c897572b6405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:26.266814 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e38a8d-49fa-43e8-afee-c897572b6405 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:26.267565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e38a8d-49fa-43e8-afee-c897572b6405 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:26.269570 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e38a8d-49fa-43e8-afee-c897572b6405 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:26.277065 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:26.415260 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1398/2788] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:43:26 2022] POST /identity/v3/users => generated 287 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:43:26.419918 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc910a7-f534-4484-94fe-14a6b9cdcfba admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:26.430698 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc910a7-f534-4484-94fe-14a6b9cdcfba admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:26.435534 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a3001-59bf-4f3e-a4dd-850b8425d357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:26.436197 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a3001-59bf-4f3e-a4dd-850b8425d357 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:26.436482 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a3001-59bf-4f3e-a4dd-850b8425d357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:26.436761 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a3001-59bf-4f3e-a4dd-850b8425d357 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:26.437498 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a3001-59bf-4f3e-a4dd-850b8425d357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:26.439131 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a3001-59bf-4f3e-a4dd-850b8425d357 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:26.450333 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1391/2789] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:43:26.454547 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e38a8d-49fa-43e8-afee-c897572b6405 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:26.465599 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e38a8d-49fa-43e8-afee-c897572b6405 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:26.470436 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e20940-edde-41bb-9a30-f6f32050241e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:26.471129 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e20940-edde-41bb-9a30-f6f32050241e admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:26.471420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e20940-edde-41bb-9a30-f6f32050241e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:26.471695 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e20940-edde-41bb-9a30-f6f32050241e admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:26.472441 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e20940-edde-41bb-9a30-f6f32050241e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:26.474465 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e20940-edde-41bb-9a30-f6f32050241e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:26.485966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1399/2790] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:43:26.490203 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a3001-59bf-4f3e-a4dd-850b8425d357 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:26.500104 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a3001-59bf-4f3e-a4dd-850b8425d357 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:26.504593 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a29814-6b61-4d3d-8328-d5813d611ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:26.505464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a29814-6b61-4d3d-8328-d5813d611ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:26.505745 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a29814-6b61-4d3d-8328-d5813d611ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:26.506018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a29814-6b61-4d3d-8328-d5813d611ef8 admin admin] PATH_INFO: `/v3/projects/a10f937788ef46399485d2b830a4e77a/users/33c3dc2a9e204e868639bd08779ea491/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:26.521018 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a29814-6b61-4d3d-8328-d5813d611ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a10f937788ef46399485d2b830a4e77a, user_id=33c3dc2a9e204e868639bd08779ea491, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:26.523720 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a29814-6b61-4d3d-8328-d5813d611ef8 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:26.628735 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1392/2791] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:43:26 2022] PUT /identity/v3/projects/a10f937788ef46399485d2b830a4e77a/users/33c3dc2a9e204e868639bd08779ea491/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:26.639241 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f278f-a808-4190-b760-dca864cb1074 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:26.639535 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f278f-a808-4190-b760-dca864cb1074 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:26.639817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f278f-a808-4190-b760-dca864cb1074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:26.666138 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-5f7f278f-a808-4190-b760-dca864cb1074 None None] MFA Rules not processed for user `33c3dc2a9e204e868639bd08779ea491`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:26.667439 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-5f7f278f-a808-4190-b760-dca864cb1074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:26.733427 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1400/2792] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:43:26.745005 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a29814-6b61-4d3d-8328-d5813d611ef8 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:26.746698 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e0a29814-6b61-4d3d-8328-d5813d611ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:26.820142 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a29814-6b61-4d3d-8328-d5813d611ef8 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:26.826773 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df39598-f1eb-4cdb-9228-2a7dc4e084d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:26.827942 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df39598-f1eb-4cdb-9228-2a7dc4e084d5 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:26.828461 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df39598-f1eb-4cdb-9228-2a7dc4e084d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:26.828944 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df39598-f1eb-4cdb-9228-2a7dc4e084d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:26.840237 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df39598-f1eb-4cdb-9228-2a7dc4e084d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:26.841826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df39598-f1eb-4cdb-9228-2a7dc4e084d5 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:26.851288 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1393/2793] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:43:30.227619 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f278f-a808-4190-b760-dca864cb1074 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:30.288904 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f278f-a808-4190-b760-dca864cb1074 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:30.294769 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ca65e-73a1-41d9-a996-22f4149ac928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:30.295648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ca65e-73a1-41d9-a996-22f4149ac928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:30.295931 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ca65e-73a1-41d9-a996-22f4149ac928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:30.296212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ca65e-73a1-41d9-a996-22f4149ac928 admin admin] PATH_INFO: `/v3/users/33c3dc2a9e204e868639bd08779ea491` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:30.297432 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ca65e-73a1-41d9-a996-22f4149ac928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33c3dc2a9e204e868639bd08779ea491)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:30.299402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ca65e-73a1-41d9-a996-22f4149ac928 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:30.520369 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-745ca65e-73a1-41d9-a996-22f4149ac928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33c3dc2a9e204e868639bd08779ea491'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:30.603109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-745ca65e-73a1-41d9-a996-22f4149ac928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33c3dc2a9e204e868639bd08779ea491'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:30.611560 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-745ca65e-73a1-41d9-a996-22f4149ac928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33c3dc2a9e204e868639bd08779ea491'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:30.618875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-745ca65e-73a1-41d9-a996-22f4149ac928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33c3dc2a9e204e868639bd08779ea491'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:30.618875 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:30.624866 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-745ca65e-73a1-41d9-a996-22f4149ac928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33c3dc2a9e204e868639bd08779ea491'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:30.631052 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1401/2794] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:43:30 2022] DELETE /identity/v3/users/33c3dc2a9e204e868639bd08779ea491 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:30.636121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4869e4-00a4-4f4d-af4d-9433e0192a1c None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:30.636628 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4869e4-00a4-4f4d-af4d-9433e0192a1c None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:30.636748 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4869e4-00a4-4f4d-af4d-9433e0192a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:30.655008 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-2d4869e4-00a4-4f4d-af4d-9433e0192a1c None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:30.676232 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-2d4869e4-00a4-4f4d-af4d-9433e0192a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:30.724930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1394/2795] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:30.735774 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ca65e-73a1-41d9-a996-22f4149ac928 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:30.740543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-745ca65e-73a1-41d9-a996-22f4149ac928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:30.799349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ca65e-73a1-41d9-a996-22f4149ac928 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:30.802899 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c02c7-ab38-4156-aa46-1891ae8367b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:30.803551 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c02c7-ab38-4156-aa46-1891ae8367b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:30.803830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c02c7-ab38-4156-aa46-1891ae8367b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:30.804110 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c02c7-ab38-4156-aa46-1891ae8367b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:30.814395 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c02c7-ab38-4156-aa46-1891ae8367b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:30.815810 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c02c7-ab38-4156-aa46-1891ae8367b6 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:30.829564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1402/2796] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:43:31.224221 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4869e4-00a4-4f4d-af4d-9433e0192a1c None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:31.225715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-2d4869e4-00a4-4f4d-af4d-9433e0192a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:31.234846 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:31.239023 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4869e4-00a4-4f4d-af4d-9433e0192a1c None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:31.244378 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa0a04-ad1b-4d02-801a-a0dbe90dc91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:31.245260 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa0a04-ad1b-4d02-801a-a0dbe90dc91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:31.245546 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa0a04-ad1b-4d02-801a-a0dbe90dc91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:31.245822 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa0a04-ad1b-4d02-801a-a0dbe90dc91f admin admin] PATH_INFO: `/v3/projects/a10f937788ef46399485d2b830a4e77a` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:31.247208 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa0a04-ad1b-4d02-801a-a0dbe90dc91f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a10f937788ef46399485d2b830a4e77a)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:31.249135 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa0a04-ad1b-4d02-801a-a0dbe90dc91f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:31.408260 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-fdfa0a04-ad1b-4d02-801a-a0dbe90dc91f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a10f937788ef46399485d2b830a4e77a', 'request_id': 'req-fdfa0a04-ad1b-4d02-801a-a0dbe90dc91f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:31.494420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-fdfa0a04-ad1b-4d02-801a-a0dbe90dc91f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a10f937788ef46399485d2b830a4e77a', 'request_id': 'req-fdfa0a04-ad1b-4d02-801a-a0dbe90dc91f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:31.503584 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-fdfa0a04-ad1b-4d02-801a-a0dbe90dc91f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a10f937788ef46399485d2b830a4e77a', 'request_id': 'req-fdfa0a04-ad1b-4d02-801a-a0dbe90dc91f'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:31.514922 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-fdfa0a04-ad1b-4d02-801a-a0dbe90dc91f admin admin] The token cache is being invalidate because project a10f937788ef46399485d2b830a4e77a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:43:31.515214 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-fdfa0a04-ad1b-4d02-801a-a0dbe90dc91f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:31.525476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-fdfa0a04-ad1b-4d02-801a-a0dbe90dc91f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:31.525476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1395/2797] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:43:31 2022] DELETE /identity/v3/projects/a10f937788ef46399485d2b830a4e77a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:31.525779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5958cd-25b2-4bf3-a4e2-945823f244da None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:31.525779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5958cd-25b2-4bf3-a4e2-945823f244da None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:31.525779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5958cd-25b2-4bf3-a4e2-945823f244da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:31.543268 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-3c5958cd-25b2-4bf3-a4e2-945823f244da None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:31.544513 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-3c5958cd-25b2-4bf3-a4e2-945823f244da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:31.606565 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1403/2798] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:43:31.616427 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6507615-ffe0-4383-8df7-222db96e8609 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:31.616933 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6507615-ffe0-4383-8df7-222db96e8609 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:31.617736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6507615-ffe0-4383-8df7-222db96e8609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:31.637639 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-b6507615-ffe0-4383-8df7-222db96e8609 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:31.638876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b6507615-ffe0-4383-8df7-222db96e8609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:31.647514 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1396/2799] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:31.652138 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5958cd-25b2-4bf3-a4e2-945823f244da None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:31.662080 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5958cd-25b2-4bf3-a4e2-945823f244da None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:31.666785 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c063a-e6e9-4fb0-bd04-dc0f4a7fc42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:31.667456 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c063a-e6e9-4fb0-bd04-dc0f4a7fc42a admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:31.667740 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c063a-e6e9-4fb0-bd04-dc0f4a7fc42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:31.668017 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c063a-e6e9-4fb0-bd04-dc0f4a7fc42a admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:31.668778 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c063a-e6e9-4fb0-bd04-dc0f4a7fc42a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:43:31.669086 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c063a-e6e9-4fb0-bd04-dc0f4a7fc42a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:31.670724 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c063a-e6e9-4fb0-bd04-dc0f4a7fc42a admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:31.683094 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1404/2800] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:43:31.687340 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6507615-ffe0-4383-8df7-222db96e8609 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:31.697341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6507615-ffe0-4383-8df7-222db96e8609 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:31.702388 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea14725-fe35-4520-bdd4-2c6fbaa37a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:31.703047 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea14725-fe35-4520-bdd4-2c6fbaa37a57 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:31.703326 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea14725-fe35-4520-bdd4-2c6fbaa37a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:31.703599 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea14725-fe35-4520-bdd4-2c6fbaa37a57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:31.704349 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea14725-fe35-4520-bdd4-2c6fbaa37a57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:31.706236 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea14725-fe35-4520-bdd4-2c6fbaa37a57 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:31.810746 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1397/2801] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:43:31 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:43:31.814956 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c063a-e6e9-4fb0-bd04-dc0f4a7fc42a admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:31.825561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c063a-e6e9-4fb0-bd04-dc0f4a7fc42a admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:31.883204 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd0a5e-6ae6-4993-90ac-7c7c9c45710b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:31.883877 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd0a5e-6ae6-4993-90ac-7c7c9c45710b admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:31.884181 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd0a5e-6ae6-4993-90ac-7c7c9c45710b admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:31.884455 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd0a5e-6ae6-4993-90ac-7c7c9c45710b admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:31.885149 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd0a5e-6ae6-4993-90ac-7c7c9c45710b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:31.887087 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd0a5e-6ae6-4993-90ac-7c7c9c45710b admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:32.063878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1405/2802] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:43:31 2022] POST /identity/v3/users => generated 286 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:43:32.068157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea14725-fe35-4520-bdd4-2c6fbaa37a57 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:32.080677 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea14725-fe35-4520-bdd4-2c6fbaa37a57 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:32.085993 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007bd17-412e-43bf-814e-40a1ee3a7ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:32.086719 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007bd17-412e-43bf-814e-40a1ee3a7ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:32.087033 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007bd17-412e-43bf-814e-40a1ee3a7ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:32.087343 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007bd17-412e-43bf-814e-40a1ee3a7ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:32.088092 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007bd17-412e-43bf-814e-40a1ee3a7ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:32.089764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007bd17-412e-43bf-814e-40a1ee3a7ca4 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:32.101938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1398/2803] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:43:32.107882 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd0a5e-6ae6-4993-90ac-7c7c9c45710b admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:32.118253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd0a5e-6ae6-4993-90ac-7c7c9c45710b admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:32.124020 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88560b34-37a1-4566-a8f5-0c70360c509d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:32.124726 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88560b34-37a1-4566-a8f5-0c70360c509d admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:32.125011 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88560b34-37a1-4566-a8f5-0c70360c509d admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:32.125288 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88560b34-37a1-4566-a8f5-0c70360c509d admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:32.126016 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88560b34-37a1-4566-a8f5-0c70360c509d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:32.127661 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88560b34-37a1-4566-a8f5-0c70360c509d admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:32.129234 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:32.139335 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1406/2804] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:43:32.148347 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007bd17-412e-43bf-814e-40a1ee3a7ca4 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:32.158658 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007bd17-412e-43bf-814e-40a1ee3a7ca4 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:32.163272 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32a02f-e109-44e1-9028-8451790015a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:32.164158 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32a02f-e109-44e1-9028-8451790015a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:32.164444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32a02f-e109-44e1-9028-8451790015a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:32.164721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32a02f-e109-44e1-9028-8451790015a1 admin admin] PATH_INFO: `/v3/projects/988e25d62eb6455c9e894bebc6263fc4/users/e1b10c7c46df4313b789b15df30cee6a/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:32.181885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32a02f-e109-44e1-9028-8451790015a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=988e25d62eb6455c9e894bebc6263fc4, user_id=e1b10c7c46df4313b789b15df30cee6a, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:32.184570 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32a02f-e109-44e1-9028-8451790015a1 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:32.265905 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1399/2805] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:43:32 2022] PUT /identity/v3/projects/988e25d62eb6455c9e894bebc6263fc4/users/e1b10c7c46df4313b789b15df30cee6a/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:32.278481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48382bc7-d2e8-462d-a198-f7a8797febb5 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:32.278718 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48382bc7-d2e8-462d-a198-f7a8797febb5 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:32.278827 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48382bc7-d2e8-462d-a198-f7a8797febb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:32.321476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-48382bc7-d2e8-462d-a198-f7a8797febb5 None None] MFA Rules not processed for user `e1b10c7c46df4313b789b15df30cee6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:32.323377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-48382bc7-d2e8-462d-a198-f7a8797febb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:32.404257 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1407/2806] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:32.417995 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32a02f-e109-44e1-9028-8451790015a1 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:32.423772 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-af32a02f-e109-44e1-9028-8451790015a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:32.500672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32a02f-e109-44e1-9028-8451790015a1 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:32.507299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7cfc2-c70f-4cc0-bc90-995d71d598a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:32.508046 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7cfc2-c70f-4cc0-bc90-995d71d598a6 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:32.508329 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7cfc2-c70f-4cc0-bc90-995d71d598a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:32.508608 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7cfc2-c70f-4cc0-bc90-995d71d598a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:32.517637 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7cfc2-c70f-4cc0-bc90-995d71d598a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:32.519267 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7cfc2-c70f-4cc0-bc90-995d71d598a6 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:32.529967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1400/2807] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:43:32.760886 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48382bc7-d2e8-462d-a198-f7a8797febb5 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:32.827212 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48382bc7-d2e8-462d-a198-f7a8797febb5 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:32.831878 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a700b-9c43-4d08-a479-16c374c5eb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:32.832751 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a700b-9c43-4d08-a479-16c374c5eb54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:32.833123 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a700b-9c43-4d08-a479-16c374c5eb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:32.833400 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a700b-9c43-4d08-a479-16c374c5eb54 admin admin] PATH_INFO: `/v3/users/e1b10c7c46df4313b789b15df30cee6a` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:32.834635 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a700b-9c43-4d08-a479-16c374c5eb54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1b10c7c46df4313b789b15df30cee6a)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:32.836680 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a700b-9c43-4d08-a479-16c374c5eb54 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:33.167422 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-673a700b-9c43-4d08-a479-16c374c5eb54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1b10c7c46df4313b789b15df30cee6a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:33.274147 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-673a700b-9c43-4d08-a479-16c374c5eb54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1b10c7c46df4313b789b15df30cee6a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:33.281877 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-673a700b-9c43-4d08-a479-16c374c5eb54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1b10c7c46df4313b789b15df30cee6a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:33.282576 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-673a700b-9c43-4d08-a479-16c374c5eb54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1b10c7c46df4313b789b15df30cee6a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:33.295809 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-673a700b-9c43-4d08-a479-16c374c5eb54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1b10c7c46df4313b789b15df30cee6a'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:33.302318 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1408/2808] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:43:32 2022] DELETE /identity/v3/users/e1b10c7c46df4313b789b15df30cee6a => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:33.307646 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deca4ae-e890-449a-a0a5-0af8a2155aff None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:33.308120 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deca4ae-e890-449a-a0a5-0af8a2155aff None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:33.308262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deca4ae-e890-449a-a0a5-0af8a2155aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:33.320715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:33.326952 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-4deca4ae-e890-449a-a0a5-0af8a2155aff None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:33.328204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4deca4ae-e890-449a-a0a5-0af8a2155aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:33.390687 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1401/2809] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:33.401736 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a700b-9c43-4d08-a479-16c374c5eb54 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:33.403185 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-673a700b-9c43-4d08-a479-16c374c5eb54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:33.409781 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:33.462160 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a700b-9c43-4d08-a479-16c374c5eb54 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:33.465939 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4e81c-773d-4a48-905e-b9750b91f530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:33.466623 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4e81c-773d-4a48-905e-b9750b91f530 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:33.466910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4e81c-773d-4a48-905e-b9750b91f530 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:33.467192 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4e81c-773d-4a48-905e-b9750b91f530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:33.476470 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4e81c-773d-4a48-905e-b9750b91f530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:33.478543 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4e81c-773d-4a48-905e-b9750b91f530 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:33.489564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1409/2810] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:43:34.104982 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deca4ae-e890-449a-a0a5-0af8a2155aff None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:34.107011 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4deca4ae-e890-449a-a0a5-0af8a2155aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:34.120321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deca4ae-e890-449a-a0a5-0af8a2155aff None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:34.125240 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c30f22-85ca-4631-9486-293bfc499d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:34.126129 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c30f22-85ca-4631-9486-293bfc499d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:34.126447 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c30f22-85ca-4631-9486-293bfc499d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:34.126736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c30f22-85ca-4631-9486-293bfc499d0c admin admin] PATH_INFO: `/v3/projects/988e25d62eb6455c9e894bebc6263fc4` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:34.127951 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c30f22-85ca-4631-9486-293bfc499d0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=988e25d62eb6455c9e894bebc6263fc4)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:34.129858 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c30f22-85ca-4631-9486-293bfc499d0c admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:34.269263 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-64c30f22-85ca-4631-9486-293bfc499d0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '988e25d62eb6455c9e894bebc6263fc4', 'request_id': 'req-64c30f22-85ca-4631-9486-293bfc499d0c'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:34.339670 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-64c30f22-85ca-4631-9486-293bfc499d0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '988e25d62eb6455c9e894bebc6263fc4', 'request_id': 'req-64c30f22-85ca-4631-9486-293bfc499d0c'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:34.346821 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-64c30f22-85ca-4631-9486-293bfc499d0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '988e25d62eb6455c9e894bebc6263fc4', 'request_id': 'req-64c30f22-85ca-4631-9486-293bfc499d0c'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:34.357967 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-64c30f22-85ca-4631-9486-293bfc499d0c admin admin] The token cache is being invalidate because project 988e25d62eb6455c9e894bebc6263fc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:43:34.358319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-64c30f22-85ca-4631-9486-293bfc499d0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:34.358995 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-64c30f22-85ca-4631-9486-293bfc499d0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:34.360561 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1402/2811] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:43:34 2022] DELETE /identity/v3/projects/988e25d62eb6455c9e894bebc6263fc4 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:34.365656 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b35747-b872-4f64-bd87-d95024683c27 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:34.366162 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b35747-b872-4f64-bd87-d95024683c27 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:34.366300 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b35747-b872-4f64-bd87-d95024683c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:34.385390 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-b4b35747-b872-4f64-bd87-d95024683c27 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:34.386662 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-b4b35747-b872-4f64-bd87-d95024683c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:34.447224 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1410/2812] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:34.455425 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8995dc-2d9b-4661-93f6-1d0cae2e880c None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:34.455716 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8995dc-2d9b-4661-93f6-1d0cae2e880c None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:34.455997 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8995dc-2d9b-4661-93f6-1d0cae2e880c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:34.474826 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-4d8995dc-2d9b-4661-93f6-1d0cae2e880c None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:34.476048 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4d8995dc-2d9b-4661-93f6-1d0cae2e880c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:34.484683 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1403/2813] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:34.489015 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b35747-b872-4f64-bd87-d95024683c27 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:34.498910 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b35747-b872-4f64-bd87-d95024683c27 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:34.503817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65204b-20e8-41a6-a6d9-6cf5aac6d395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:34.504481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65204b-20e8-41a6-a6d9-6cf5aac6d395 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:34.504772 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65204b-20e8-41a6-a6d9-6cf5aac6d395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:34.505074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65204b-20e8-41a6-a6d9-6cf5aac6d395 admin admin] PATH_INFO: `/v3/domains` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:34.505809 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65204b-20e8-41a6-a6d9-6cf5aac6d395 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:43:34.506116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65204b-20e8-41a6-a6d9-6cf5aac6d395 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:34.507780 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65204b-20e8-41a6-a6d9-6cf5aac6d395 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:34.519937 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1411/2814] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:43:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:43:34.528867 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8995dc-2d9b-4661-93f6-1d0cae2e880c None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:34.538521 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8995dc-2d9b-4661-93f6-1d0cae2e880c None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:34.544229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a673b28-a2db-49c8-bc9f-b9dfa6ab93c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:34.544878 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a673b28-a2db-49c8-bc9f-b9dfa6ab93c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:34.545189 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a673b28-a2db-49c8-bc9f-b9dfa6ab93c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:34.545464 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a673b28-a2db-49c8-bc9f-b9dfa6ab93c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:34.546173 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a673b28-a2db-49c8-bc9f-b9dfa6ab93c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:34.548085 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a673b28-a2db-49c8-bc9f-b9dfa6ab93c4 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:34.661262 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1404/2815] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:43:34 2022] POST /identity/v3/projects => generated 381 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:43:34.666074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65204b-20e8-41a6-a6d9-6cf5aac6d395 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:34.676368 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65204b-20e8-41a6-a6d9-6cf5aac6d395 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:34.733520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b4194-dee4-479d-9195-55cdc9535644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:34.734193 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b4194-dee4-479d-9195-55cdc9535644 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:34.734635 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b4194-dee4-479d-9195-55cdc9535644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:34.734914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b4194-dee4-479d-9195-55cdc9535644 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:34.735692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b4194-dee4-479d-9195-55cdc9535644 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:34.737604 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b4194-dee4-479d-9195-55cdc9535644 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:34.863080 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1412/2816] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:43:34 2022] POST /identity/v3/users => generated 291 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:43:34.867188 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a673b28-a2db-49c8-bc9f-b9dfa6ab93c4 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:34.877804 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a673b28-a2db-49c8-bc9f-b9dfa6ab93c4 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:34.882461 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ed25f-6273-4a8c-9fd5-c15d6298a15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:34.883152 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ed25f-6273-4a8c-9fd5-c15d6298a15a admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:34.883441 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ed25f-6273-4a8c-9fd5-c15d6298a15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:34.883782 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ed25f-6273-4a8c-9fd5-c15d6298a15a admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:34.885013 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ed25f-6273-4a8c-9fd5-c15d6298a15a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:34.886655 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ed25f-6273-4a8c-9fd5-c15d6298a15a admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:34.897102 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1405/2817] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:43:34.901801 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b4194-dee4-479d-9195-55cdc9535644 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:34.913214 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b4194-dee4-479d-9195-55cdc9535644 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:34.916874 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636eefe-be0a-41bc-8650-83e814abd4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:34.917529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636eefe-be0a-41bc-8650-83e814abd4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:34.917808 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636eefe-be0a-41bc-8650-83e814abd4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:34.918079 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636eefe-be0a-41bc-8650-83e814abd4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:34.918905 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636eefe-be0a-41bc-8650-83e814abd4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:34.920484 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636eefe-be0a-41bc-8650-83e814abd4d4 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:34.930628 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1413/2818] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 15:43:34.934715 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ed25f-6273-4a8c-9fd5-c15d6298a15a admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:34.945318 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ed25f-6273-4a8c-9fd5-c15d6298a15a admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:34.950991 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecc7e6-1a32-46f7-8f9c-e4d8a6307052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:34.951919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecc7e6-1a32-46f7-8f9c-e4d8a6307052 admin admin] REQUEST_METHOD: `PUT` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:34.952200 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecc7e6-1a32-46f7-8f9c-e4d8a6307052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:34.952479 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecc7e6-1a32-46f7-8f9c-e4d8a6307052 admin admin] PATH_INFO: `/v3/projects/90655a29aa1c403dac18e115e19f61a1/users/812971869c274f62a4089c2c9230d8ec/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:34.968291 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ecc7e6-1a32-46f7-8f9c-e4d8a6307052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90655a29aa1c403dac18e115e19f61a1, user_id=812971869c274f62a4089c2c9230d8ec, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:34.970991 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ecc7e6-1a32-46f7-8f9c-e4d8a6307052 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:35.056462 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1406/2819] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:43:34 2022] PUT /identity/v3/projects/90655a29aa1c403dac18e115e19f61a1/users/812971869c274f62a4089c2c9230d8ec/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:35.067064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6e723-2624-4704-814a-1755bffc71f5 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:35.067349 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6e723-2624-4704-814a-1755bffc71f5 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:35.067628 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6e723-2624-4704-814a-1755bffc71f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:35.092842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:35.096440 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-f6a6e723-2624-4704-814a-1755bffc71f5 None None] MFA Rules not processed for user `812971869c274f62a4089c2c9230d8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:35.097670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f6a6e723-2624-4704-814a-1755bffc71f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:35.165638 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1414/2820] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:35.170163 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecc7e6-1a32-46f7-8f9c-e4d8a6307052 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:35.201793 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:35.236159 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecc7e6-1a32-46f7-8f9c-e4d8a6307052 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:35.241420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2071b-9a0b-4fd1-bc85-5677b64fd586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:35.242077 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2071b-9a0b-4fd1-bc85-5677b64fd586 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:35.242399 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2071b-9a0b-4fd1-bc85-5677b64fd586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:35.242681 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2071b-9a0b-4fd1-bc85-5677b64fd586 admin admin] PATH_INFO: `/v3/projects` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:35.243889 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2071b-9a0b-4fd1-bc85-5677b64fd586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:35.245834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2071b-9a0b-4fd1-bc85-5677b64fd586 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:35.364356 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1407/2821] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:43:35 2022] POST /identity/v3/projects => generated 381 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:43:35.369249 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6e723-2624-4704-814a-1755bffc71f5 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:35.380520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6e723-2624-4704-814a-1755bffc71f5 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:35.438804 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fe3c8-4317-4eaa-a004-165535e0b323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:35.439420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fe3c8-4317-4eaa-a004-165535e0b323 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:35.439704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fe3c8-4317-4eaa-a004-165535e0b323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:35.439980 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fe3c8-4317-4eaa-a004-165535e0b323 admin admin] PATH_INFO: `/v3/users` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:35.440685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fe3c8-4317-4eaa-a004-165535e0b323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:35.442631 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fe3c8-4317-4eaa-a004-165535e0b323 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:35.563312 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1415/2822] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:43:35 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:43:35.567899 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2071b-9a0b-4fd1-bc85-5677b64fd586 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:35.578815 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2071b-9a0b-4fd1-bc85-5677b64fd586 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:35.583614 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c854a-33c1-4e63-b0d2-a57c1341278b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:35.584268 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c854a-33c1-4e63-b0d2-a57c1341278b admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:35.584581 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c854a-33c1-4e63-b0d2-a57c1341278b admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:35.584862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c854a-33c1-4e63-b0d2-a57c1341278b admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:35.585582 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c854a-33c1-4e63-b0d2-a57c1341278b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:35.587989 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c854a-33c1-4e63-b0d2-a57c1341278b admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:35.599321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1408/2823] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:43:35.610546 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fe3c8-4317-4eaa-a004-165535e0b323 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:35.620615 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fe3c8-4317-4eaa-a004-165535e0b323 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:35.625057 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f298f12-4395-4e64-a225-bad51d20f97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:35.626001 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f298f12-4395-4e64-a225-bad51d20f97b admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:35.626309 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f298f12-4395-4e64-a225-bad51d20f97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:35.626590 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f298f12-4395-4e64-a225-bad51d20f97b admin admin] PATH_INFO: `/v3/projects/8cebe43273614113b10e4058bf33ee21/users/0f0a324e567748efbc97bf8914aa3e80/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:35.641834 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f298f12-4395-4e64-a225-bad51d20f97b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cebe43273614113b10e4058bf33ee21, user_id=0f0a324e567748efbc97bf8914aa3e80, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:35.644523 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f298f12-4395-4e64-a225-bad51d20f97b admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:35.725483 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1416/2824] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:43:35 2022] PUT /identity/v3/projects/8cebe43273614113b10e4058bf33ee21/users/0f0a324e567748efbc97bf8914aa3e80/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:35.729563 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c854a-33c1-4e63-b0d2-a57c1341278b admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:35.740050 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c854a-33c1-4e63-b0d2-a57c1341278b admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:35.798925 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb3b04-46c3-4a80-b411-2bac733582cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:35.799599 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb3b04-46c3-4a80-b411-2bac733582cc admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:35.799885 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb3b04-46c3-4a80-b411-2bac733582cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:35.800164 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb3b04-46c3-4a80-b411-2bac733582cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:35.800888 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb3b04-46c3-4a80-b411-2bac733582cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:35.802499 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb3b04-46c3-4a80-b411-2bac733582cc admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:35.812430 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1409/2825] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:43:35.816646 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f298f12-4395-4e64-a225-bad51d20f97b admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:35.827722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f298f12-4395-4e64-a225-bad51d20f97b admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:35.832475 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d696b-66cf-458d-88ff-a4aef83471c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:35.836779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d696b-66cf-458d-88ff-a4aef83471c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:35.836779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d696b-66cf-458d-88ff-a4aef83471c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:35.836779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d696b-66cf-458d-88ff-a4aef83471c6 admin admin] PATH_INFO: `/v3/projects/8cebe43273614113b10e4058bf33ee21/users/0f0a324e567748efbc97bf8914aa3e80/roles/25516be1485c49538544ea8a596e7d17` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:35.836779 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d696b-66cf-458d-88ff-a4aef83471c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cebe43273614113b10e4058bf33ee21, user_id=0f0a324e567748efbc97bf8914aa3e80, role_id=25516be1485c49538544ea8a596e7d17)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:35.839227 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d696b-66cf-458d-88ff-a4aef83471c6 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:35.849571 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1417/2826] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:43:35 2022] PUT /identity/v3/projects/8cebe43273614113b10e4058bf33ee21/users/0f0a324e567748efbc97bf8914aa3e80/roles/25516be1485c49538544ea8a596e7d17 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:35.912420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6089a8a-0e07-415e-8171-08a755873f8b None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:35.913004 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6089a8a-0e07-415e-8171-08a755873f8b None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:35.913004 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6089a8a-0e07-415e-8171-08a755873f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:35.939369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-d6089a8a-0e07-415e-8171-08a755873f8b None None] MFA Rules not processed for user `0f0a324e567748efbc97bf8914aa3e80`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:35.940680 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d6089a8a-0e07-415e-8171-08a755873f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:36.010920 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1410/2827] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:36.023126 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d696b-66cf-458d-88ff-a4aef83471c6 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:36.024896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-4d3d696b-66cf-458d-88ff-a4aef83471c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:36.083215 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d696b-66cf-458d-88ff-a4aef83471c6 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:36.087028 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bad43-14be-438d-a31a-35e030a52a84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:36.087678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bad43-14be-438d-a31a-35e030a52a84 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:36.087971 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bad43-14be-438d-a31a-35e030a52a84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:36.088249 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bad43-14be-438d-a31a-35e030a52a84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:36.098103 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5bad43-14be-438d-a31a-35e030a52a84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:36.099583 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5bad43-14be-438d-a31a-35e030a52a84 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:36.103722 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:36.109943 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1418/2828] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:43:37.321899 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6089a8a-0e07-415e-8171-08a755873f8b None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:37.331351 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6089a8a-0e07-415e-8171-08a755873f8b None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:37.335943 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689f202-2cb7-4751-9a4e-69c6893b7b0f tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0a324e567748efbc97bf8914aa3e80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8cebe43273614113b10e4058bf33ee21', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:37.336599 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689f202-2cb7-4751-9a4e-69c6893b7b0f tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:37.336889 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689f202-2cb7-4751-9a4e-69c6893b7b0f tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:37.337745 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689f202-2cb7-4751-9a4e-69c6893b7b0f tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:37.338544 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689f202-2cb7-4751-9a4e-69c6893b7b0f tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:43:37.338862 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689f202-2cb7-4751-9a4e-69c6893b7b0f tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:37.340445 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689f202-2cb7-4751-9a4e-69c6893b7b0f tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:37.352078 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1411/2829] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:43:37.356197 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bad43-14be-438d-a31a-35e030a52a84 service cinder] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:37.365191 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bad43-14be-438d-a31a-35e030a52a84 service cinder] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:37.369807 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136294f3-6f28-4bf7-999c-f97add08d42b tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0a324e567748efbc97bf8914aa3e80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8cebe43273614113b10e4058bf33ee21', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:37.370495 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136294f3-6f28-4bf7-999c-f97add08d42b tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:37.370783 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136294f3-6f28-4bf7-999c-f97add08d42b tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:37.371060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136294f3-6f28-4bf7-999c-f97add08d42b tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:37.371761 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136294f3-6f28-4bf7-999c-f97add08d42b tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:37.373677 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136294f3-6f28-4bf7-999c-f97add08d42b tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:37.472653 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1419/2830] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:43:37 2022] POST /identity/v3/projects => generated 355 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:43:37.476834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689f202-2cb7-4751-9a4e-69c6893b7b0f tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:37.487446 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689f202-2cb7-4751-9a4e-69c6893b7b0f tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:37.493673 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:37.546192 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d40958-f06e-4f5d-b832-79e39f2d0d98 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0a324e567748efbc97bf8914aa3e80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8cebe43273614113b10e4058bf33ee21', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:37.546810 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d40958-f06e-4f5d-b832-79e39f2d0d98 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:37.547117 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d40958-f06e-4f5d-b832-79e39f2d0d98 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:37.547376 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d40958-f06e-4f5d-b832-79e39f2d0d98 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:37.548106 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d40958-f06e-4f5d-b832-79e39f2d0d98 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:43:37.548412 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d40958-f06e-4f5d-b832-79e39f2d0d98 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:37.550060 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d40958-f06e-4f5d-b832-79e39f2d0d98 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:37.563793 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1412/2831] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:43:37.601057 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136294f3-6f28-4bf7-999c-f97add08d42b tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:37.611783 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136294f3-6f28-4bf7-999c-f97add08d42b tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:37.616498 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e077d5e-365b-4ab9-a72a-90d3bafe6f50 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0a324e567748efbc97bf8914aa3e80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8cebe43273614113b10e4058bf33ee21', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:37.617423 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e077d5e-365b-4ab9-a72a-90d3bafe6f50 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:37.617744 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e077d5e-365b-4ab9-a72a-90d3bafe6f50 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:37.618032 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e077d5e-365b-4ab9-a72a-90d3bafe6f50 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] PATH_INFO: `/v3/projects/bbe4ad055a9d429693bba189550d28f2` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:37.619800 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e077d5e-365b-4ab9-a72a-90d3bafe6f50 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bbe4ad055a9d429693bba189550d28f2)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:37.621851 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e077d5e-365b-4ab9-a72a-90d3bafe6f50 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:37.736840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-3e077d5e-365b-4ab9-a72a-90d3bafe6f50 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbe4ad055a9d429693bba189550d28f2', 'request_id': 'req-3e077d5e-365b-4ab9-a72a-90d3bafe6f50'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:37.822645 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-3e077d5e-365b-4ab9-a72a-90d3bafe6f50 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbe4ad055a9d429693bba189550d28f2', 'request_id': 'req-3e077d5e-365b-4ab9-a72a-90d3bafe6f50'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:37.830629 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-3e077d5e-365b-4ab9-a72a-90d3bafe6f50 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbe4ad055a9d429693bba189550d28f2', 'request_id': 'req-3e077d5e-365b-4ab9-a72a-90d3bafe6f50'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:37.842561 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-3e077d5e-365b-4ab9-a72a-90d3bafe6f50 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] The token cache is being invalidate because project bbe4ad055a9d429693bba189550d28f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:43:37.842853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-3e077d5e-365b-4ab9-a72a-90d3bafe6f50 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:37.843619 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-3e077d5e-365b-4ab9-a72a-90d3bafe6f50 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:37.845288 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1420/2832] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:43:37 2022] DELETE /identity/v3/projects/bbe4ad055a9d429693bba189550d28f2 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:38.820116 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d40958-f06e-4f5d-b832-79e39f2d0d98 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:38.822470 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e3d40958-f06e-4f5d-b832-79e39f2d0d98 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:38.898664 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d40958-f06e-4f5d-b832-79e39f2d0d98 tempest-VolumeQuotaClassesTest-2094847610 tempest-VolumeQuotaClassesTest-2094847610-project-admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:38.959083 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc94b65-ccbd-4799-b2e2-54c70eec4c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:38.959971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc94b65-ccbd-4799-b2e2-54c70eec4c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:38.960257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc94b65-ccbd-4799-b2e2-54c70eec4c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:38.960532 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc94b65-ccbd-4799-b2e2-54c70eec4c2d admin admin] PATH_INFO: `/v3/users/812971869c274f62a4089c2c9230d8ec` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:38.961764 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc94b65-ccbd-4799-b2e2-54c70eec4c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=812971869c274f62a4089c2c9230d8ec)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:38.963731 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc94b65-ccbd-4799-b2e2-54c70eec4c2d admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:39.422951 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-efc94b65-ccbd-4799-b2e2-54c70eec4c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '812971869c274f62a4089c2c9230d8ec'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:39.426655 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:39.605015 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-efc94b65-ccbd-4799-b2e2-54c70eec4c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '812971869c274f62a4089c2c9230d8ec'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:39.605733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-efc94b65-ccbd-4799-b2e2-54c70eec4c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '812971869c274f62a4089c2c9230d8ec'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:39.619587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-efc94b65-ccbd-4799-b2e2-54c70eec4c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '812971869c274f62a4089c2c9230d8ec'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:39.666005 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-efc94b65-ccbd-4799-b2e2-54c70eec4c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '812971869c274f62a4089c2c9230d8ec'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:39.666005 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1413/2833] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:43:38 2022] DELETE /identity/v3/users/812971869c274f62a4089c2c9230d8ec => generated 0 bytes in 814 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:39.666332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5b058-26f3-41ae-a4f6-7d2f44a8c8b5 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:39.666332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5b058-26f3-41ae-a4f6-7d2f44a8c8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:39.666332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5b058-26f3-41ae-a4f6-7d2f44a8c8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:39.666332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-05e5b058-26f3-41ae-a4f6-7d2f44a8c8b5 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:39.666332 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-05e5b058-26f3-41ae-a4f6-7d2f44a8c8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:39.734473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1421/2834] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:39.743000 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc94b65-ccbd-4799-b2e2-54c70eec4c2d admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:39.744579 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-efc94b65-ccbd-4799-b2e2-54c70eec4c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:39.804484 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc94b65-ccbd-4799-b2e2-54c70eec4c2d admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:39.808175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7dc11-3fc4-47f2-92ab-954f9ca48a02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:39.808893 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7dc11-3fc4-47f2-92ab-954f9ca48a02 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:39.809121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7dc11-3fc4-47f2-92ab-954f9ca48a02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:39.809394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7dc11-3fc4-47f2-92ab-954f9ca48a02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:39.821795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7dc11-3fc4-47f2-92ab-954f9ca48a02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:39.823704 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7dc11-3fc4-47f2-92ab-954f9ca48a02 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:39.834933 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1414/2835] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:43:40.287210 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5b058-26f3-41ae-a4f6-7d2f44a8c8b5 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:40.288803 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-05e5b058-26f3-41ae-a4f6-7d2f44a8c8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:40.302822 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5b058-26f3-41ae-a4f6-7d2f44a8c8b5 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:40.309641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3770db6-9417-4b05-81ab-ef5b04c64f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:40.310557 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3770db6-9417-4b05-81ab-ef5b04c64f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:40.310842 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3770db6-9417-4b05-81ab-ef5b04c64f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:40.311113 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3770db6-9417-4b05-81ab-ef5b04c64f9b admin admin] PATH_INFO: `/v3/projects/90655a29aa1c403dac18e115e19f61a1` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:40.312493 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3770db6-9417-4b05-81ab-ef5b04c64f9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=90655a29aa1c403dac18e115e19f61a1)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:40.314592 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3770db6-9417-4b05-81ab-ef5b04c64f9b admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:40.457050 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-d3770db6-9417-4b05-81ab-ef5b04c64f9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90655a29aa1c403dac18e115e19f61a1', 'request_id': 'req-d3770db6-9417-4b05-81ab-ef5b04c64f9b'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:40.460726 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=85,keystone:INSERT=3,keystone:DELETE=12 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:40.531064 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-d3770db6-9417-4b05-81ab-ef5b04c64f9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90655a29aa1c403dac18e115e19f61a1', 'request_id': 'req-d3770db6-9417-4b05-81ab-ef5b04c64f9b'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:40.537851 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-d3770db6-9417-4b05-81ab-ef5b04c64f9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90655a29aa1c403dac18e115e19f61a1', 'request_id': 'req-d3770db6-9417-4b05-81ab-ef5b04c64f9b'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:40.549102 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-d3770db6-9417-4b05-81ab-ef5b04c64f9b admin admin] The token cache is being invalidate because project 90655a29aa1c403dac18e115e19f61a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:43:40.549395 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-d3770db6-9417-4b05-81ab-ef5b04c64f9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:40.550179 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-d3770db6-9417-4b05-81ab-ef5b04c64f9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:40.551795 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1422/2836] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:43:40 2022] DELETE /identity/v3/projects/90655a29aa1c403dac18e115e19f61a1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:40.561296 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7dc11-3fc4-47f2-92ab-954f9ca48a02 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:40.572833 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7dc11-3fc4-47f2-92ab-954f9ca48a02 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:40.632943 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df101374-5d17-461d-8851-d168c4ce5db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:40.634458 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df101374-5d17-461d-8851-d168c4ce5db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:40.634743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df101374-5d17-461d-8851-d168c4ce5db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:40.635019 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df101374-5d17-461d-8851-d168c4ce5db5 admin admin] PATH_INFO: `/v3/users/0f0a324e567748efbc97bf8914aa3e80` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:40.636204 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df101374-5d17-461d-8851-d168c4ce5db5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f0a324e567748efbc97bf8914aa3e80)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:40.638110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df101374-5d17-461d-8851-d168c4ce5db5 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:40.847593 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-df101374-5d17-461d-8851-d168c4ce5db5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f0a324e567748efbc97bf8914aa3e80'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:40.925010 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-df101374-5d17-461d-8851-d168c4ce5db5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f0a324e567748efbc97bf8914aa3e80'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:40.925617 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-df101374-5d17-461d-8851-d168c4ce5db5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f0a324e567748efbc97bf8914aa3e80'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:40.938925 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-df101374-5d17-461d-8851-d168c4ce5db5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f0a324e567748efbc97bf8914aa3e80'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:40.940977 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:40.946493 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-df101374-5d17-461d-8851-d168c4ce5db5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f0a324e567748efbc97bf8914aa3e80'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:40.952738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1415/2837] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:43:40 2022] DELETE /identity/v3/users/0f0a324e567748efbc97bf8914aa3e80 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:41.432301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3770db6-9417-4b05-81ab-ef5b04c64f9b admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:41.432301 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d3770db6-9417-4b05-81ab-ef5b04c64f9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:41.444977 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3770db6-9417-4b05-81ab-ef5b04c64f9b admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:41.512758 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d79cc4-f628-4378-9c55-4663d27aed09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:41.514254 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d79cc4-f628-4378-9c55-4663d27aed09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:41.514563 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d79cc4-f628-4378-9c55-4663d27aed09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:41.514839 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d79cc4-f628-4378-9c55-4663d27aed09 admin admin] PATH_INFO: `/v3/projects/8cebe43273614113b10e4058bf33ee21` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:41.516126 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d79cc4-f628-4378-9c55-4663d27aed09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cebe43273614113b10e4058bf33ee21)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:41.518100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d79cc4-f628-4378-9c55-4663d27aed09 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:41.636572 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b8d79cc4-f628-4378-9c55-4663d27aed09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cebe43273614113b10e4058bf33ee21', 'request_id': 'req-b8d79cc4-f628-4378-9c55-4663d27aed09'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:41.776712 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b8d79cc4-f628-4378-9c55-4663d27aed09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cebe43273614113b10e4058bf33ee21', 'request_id': 'req-b8d79cc4-f628-4378-9c55-4663d27aed09'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:41.783621 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b8d79cc4-f628-4378-9c55-4663d27aed09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cebe43273614113b10e4058bf33ee21', 'request_id': 'req-b8d79cc4-f628-4378-9c55-4663d27aed09'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:41.793963 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b8d79cc4-f628-4378-9c55-4663d27aed09 admin admin] The token cache is being invalidate because project 8cebe43273614113b10e4058bf33ee21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:43:41.794267 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b8d79cc4-f628-4378-9c55-4663d27aed09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:41.794927 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b8d79cc4-f628-4378-9c55-4663d27aed09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:41.796472 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1423/2838] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:43:41 2022] DELETE /identity/v3/projects/8cebe43273614113b10e4058bf33ee21 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:41.802791 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7668f-b539-4d3c-80cc-1517fdb82b19 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:41.803081 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7668f-b539-4d3c-80cc-1517fdb82b19 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:41.803359 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7668f-b539-4d3c-80cc-1517fdb82b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:41.822000 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-b5e7668f-b539-4d3c-80cc-1517fdb82b19 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:41.823473 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-b5e7668f-b539-4d3c-80cc-1517fdb82b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:41.884478 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1416/2839] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:41.892274 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3210d-0bd4-445a-a543-a9224614c764 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:41.892564 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3210d-0bd4-445a-a543-a9224614c764 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:41.892840 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3210d-0bd4-445a-a543-a9224614c764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:41.911976 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-85e3210d-0bd4-445a-a543-a9224614c764 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:41.913186 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-85e3210d-0bd4-445a-a543-a9224614c764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:41.921244 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1424/2840] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:41.925713 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7668f-b539-4d3c-80cc-1517fdb82b19 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:41.935971 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7668f-b539-4d3c-80cc-1517fdb82b19 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:41.940640 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30651f12-9988-4e5f-bffb-0a780f5e2892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:41.944633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30651f12-9988-4e5f-bffb-0a780f5e2892 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:41.944633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30651f12-9988-4e5f-bffb-0a780f5e2892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:41.944633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30651f12-9988-4e5f-bffb-0a780f5e2892 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:41.944633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30651f12-9988-4e5f-bffb-0a780f5e2892 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:43:41.944633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30651f12-9988-4e5f-bffb-0a780f5e2892 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:41.944633 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30651f12-9988-4e5f-bffb-0a780f5e2892 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:41.956938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1417/2841] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:43:41.961265 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3210d-0bd4-445a-a543-a9224614c764 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:41.971177 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3210d-0bd4-445a-a543-a9224614c764 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:41.976026 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c963e99-e7d6-41b8-840e-5b1e8b48b3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:41.976681 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c963e99-e7d6-41b8-840e-5b1e8b48b3c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:41.976962 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c963e99-e7d6-41b8-840e-5b1e8b48b3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:41.977238 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c963e99-e7d6-41b8-840e-5b1e8b48b3c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:41.977974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c963e99-e7d6-41b8-840e-5b1e8b48b3c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:41.979914 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c963e99-e7d6-41b8-840e-5b1e8b48b3c6 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:42.158820 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1425/2842] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:43:41 2022] POST /identity/v3/projects => generated 371 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:43:42.162834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30651f12-9988-4e5f-bffb-0a780f5e2892 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:42.173000 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30651f12-9988-4e5f-bffb-0a780f5e2892 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:42.229595 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79202b11-30c5-4fd1-8e8e-6bc8f13cf420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:42.230278 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79202b11-30c5-4fd1-8e8e-6bc8f13cf420 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:42.230602 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79202b11-30c5-4fd1-8e8e-6bc8f13cf420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:42.230878 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79202b11-30c5-4fd1-8e8e-6bc8f13cf420 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:42.231539 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79202b11-30c5-4fd1-8e8e-6bc8f13cf420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:42.233426 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79202b11-30c5-4fd1-8e8e-6bc8f13cf420 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:42.339508 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1418/2843] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:43:42 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:43:42.343776 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c963e99-e7d6-41b8-840e-5b1e8b48b3c6 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:42.355010 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c963e99-e7d6-41b8-840e-5b1e8b48b3c6 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:42.359756 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eaa7a1-0084-494a-8ea0-0ab9c1ce6fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:42.360427 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eaa7a1-0084-494a-8ea0-0ab9c1ce6fac admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:42.360712 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eaa7a1-0084-494a-8ea0-0ab9c1ce6fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:42.360990 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eaa7a1-0084-494a-8ea0-0ab9c1ce6fac admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:42.361717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eaa7a1-0084-494a-8ea0-0ab9c1ce6fac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:42.363408 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eaa7a1-0084-494a-8ea0-0ab9c1ce6fac admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:42.374643 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1426/2844] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:43:42.378763 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79202b11-30c5-4fd1-8e8e-6bc8f13cf420 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:42.388672 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79202b11-30c5-4fd1-8e8e-6bc8f13cf420 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:42.393145 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8cd611-5742-41d0-a1dc-f284d5c89e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:42.393800 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8cd611-5742-41d0-a1dc-f284d5c89e66 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:42.394331 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8cd611-5742-41d0-a1dc-f284d5c89e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:42.394470 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8cd611-5742-41d0-a1dc-f284d5c89e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:42.395167 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8cd611-5742-41d0-a1dc-f284d5c89e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:42.396743 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8cd611-5742-41d0-a1dc-f284d5c89e66 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:42.406571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1419/2845] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:43:42.410648 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eaa7a1-0084-494a-8ea0-0ab9c1ce6fac admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:42.421325 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eaa7a1-0084-494a-8ea0-0ab9c1ce6fac admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:42.425860 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ae2b7-b48e-4fd5-9cbb-55839afd2ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:42.426830 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ae2b7-b48e-4fd5-9cbb-55839afd2ced admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:42.427111 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ae2b7-b48e-4fd5-9cbb-55839afd2ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:42.427380 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ae2b7-b48e-4fd5-9cbb-55839afd2ced admin admin] PATH_INFO: `/v3/projects/27e529dc8ce44eabaca72e33d70f4993/users/202da274939c40919d741a77c364747e/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:42.443065 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ae2b7-b48e-4fd5-9cbb-55839afd2ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=27e529dc8ce44eabaca72e33d70f4993, user_id=202da274939c40919d741a77c364747e, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:42.449047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ae2b7-b48e-4fd5-9cbb-55839afd2ced admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:42.568558 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1427/2846] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:43:42 2022] PUT /identity/v3/projects/27e529dc8ce44eabaca72e33d70f4993/users/202da274939c40919d741a77c364747e/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:43:42.579299 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb14e5-ce8b-4eb0-aec6-674f81e6da56 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:42.579592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb14e5-ce8b-4eb0-aec6-674f81e6da56 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:42.579876 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb14e5-ce8b-4eb0-aec6-674f81e6da56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:42.608058 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-02eb14e5-ce8b-4eb0-aec6-674f81e6da56 None None] MFA Rules not processed for user `202da274939c40919d741a77c364747e`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:42.609206 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-02eb14e5-ce8b-4eb0-aec6-674f81e6da56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:42.653731 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:42.673984 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1420/2847] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:42.686694 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ae2b7-b48e-4fd5-9cbb-55839afd2ced admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:42.688401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-558ae2b7-b48e-4fd5-9cbb-55839afd2ced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:42.699580 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:42.745026 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ae2b7-b48e-4fd5-9cbb-55839afd2ced admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:42.748721 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546614f-d6a6-4d9b-ae70-4ec8ff022efb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:42.749361 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546614f-d6a6-4d9b-ae70-4ec8ff022efb service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:42.749643 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546614f-d6a6-4d9b-ae70-4ec8ff022efb service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:42.749917 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546614f-d6a6-4d9b-ae70-4ec8ff022efb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:42.759027 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546614f-d6a6-4d9b-ae70-4ec8ff022efb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:42.760449 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546614f-d6a6-4d9b-ae70-4ec8ff022efb service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:42.769095 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1428/2848] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:43:42.788858 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb14e5-ce8b-4eb0-aec6-674f81e6da56 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:42.852121 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb14e5-ce8b-4eb0-aec6-674f81e6da56 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:42.857369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83726b-2dd6-43f8-9013-4751719b6ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:42.858353 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83726b-2dd6-43f8-9013-4751719b6ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:42.858579 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83726b-2dd6-43f8-9013-4751719b6ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:42.858858 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83726b-2dd6-43f8-9013-4751719b6ede admin admin] PATH_INFO: `/v3/users/202da274939c40919d741a77c364747e` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:42.860060 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83726b-2dd6-43f8-9013-4751719b6ede admin admin] RBAC: Authorizing `identity:delete_user(user_id=202da274939c40919d741a77c364747e)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:42.861962 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83726b-2dd6-43f8-9013-4751719b6ede admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:43.148698 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ab83726b-2dd6-43f8-9013-4751719b6ede admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '202da274939c40919d741a77c364747e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:43.248125 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ab83726b-2dd6-43f8-9013-4751719b6ede admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '202da274939c40919d741a77c364747e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:43.248757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ab83726b-2dd6-43f8-9013-4751719b6ede admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '202da274939c40919d741a77c364747e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:43.262783 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ab83726b-2dd6-43f8-9013-4751719b6ede admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '202da274939c40919d741a77c364747e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:43.270496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ab83726b-2dd6-43f8-9013-4751719b6ede admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '202da274939c40919d741a77c364747e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:43.277502 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1421/2849] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:43:42 2022] DELETE /identity/v3/users/202da274939c40919d741a77c364747e => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:43.282313 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a2afb-127f-4386-80b7-1a08a7eaa022 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:43.282605 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a2afb-127f-4386-80b7-1a08a7eaa022 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:43.282885 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a2afb-127f-4386-80b7-1a08a7eaa022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:43.301894 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-757a2afb-127f-4386-80b7-1a08a7eaa022 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:43.303467 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-757a2afb-127f-4386-80b7-1a08a7eaa022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:43.365417 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1429/2850] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:43.375831 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83726b-2dd6-43f8-9013-4751719b6ede admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:43.377341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ab83726b-2dd6-43f8-9013-4751719b6ede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:43.436863 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83726b-2dd6-43f8-9013-4751719b6ede admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:43.440419 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be5235-49f9-4eb7-b85d-4d4d1e367244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:43.441076 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be5235-49f9-4eb7-b85d-4d4d1e367244 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:43.441359 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be5235-49f9-4eb7-b85d-4d4d1e367244 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:43.441636 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be5235-49f9-4eb7-b85d-4d4d1e367244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:43.452395 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be5235-49f9-4eb7-b85d-4d4d1e367244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:43.453822 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be5235-49f9-4eb7-b85d-4d4d1e367244 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:43.464431 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1422/2851] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:43:43.921501 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a2afb-127f-4386-80b7-1a08a7eaa022 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:43.923036 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-757a2afb-127f-4386-80b7-1a08a7eaa022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:43.937334 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a2afb-127f-4386-80b7-1a08a7eaa022 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:43.943389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17695bf-0920-4110-8532-308fb8de4aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:43.944358 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17695bf-0920-4110-8532-308fb8de4aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:43.944641 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17695bf-0920-4110-8532-308fb8de4aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:43.944919 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17695bf-0920-4110-8532-308fb8de4aff admin admin] PATH_INFO: `/v3/projects/27e529dc8ce44eabaca72e33d70f4993` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:43.946307 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17695bf-0920-4110-8532-308fb8de4aff admin admin] RBAC: Authorizing `identity:delete_project(project_id=27e529dc8ce44eabaca72e33d70f4993)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:43.948626 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17695bf-0920-4110-8532-308fb8de4aff admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:44.070729 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e17695bf-0920-4110-8532-308fb8de4aff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27e529dc8ce44eabaca72e33d70f4993', 'request_id': 'req-e17695bf-0920-4110-8532-308fb8de4aff'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:44.145506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e17695bf-0920-4110-8532-308fb8de4aff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27e529dc8ce44eabaca72e33d70f4993', 'request_id': 'req-e17695bf-0920-4110-8532-308fb8de4aff'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:44.152817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e17695bf-0920-4110-8532-308fb8de4aff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27e529dc8ce44eabaca72e33d70f4993', 'request_id': 'req-e17695bf-0920-4110-8532-308fb8de4aff'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:44.163794 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e17695bf-0920-4110-8532-308fb8de4aff admin admin] The token cache is being invalidate because project 27e529dc8ce44eabaca72e33d70f4993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:43:44.164090 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e17695bf-0920-4110-8532-308fb8de4aff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:44.165268 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-e17695bf-0920-4110-8532-308fb8de4aff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:44.167080 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1430/2852] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:43:43 2022] DELETE /identity/v3/projects/27e529dc8ce44eabaca72e33d70f4993 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:44.172181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55243858-48d4-4764-919b-43eefb040dda None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:44.172957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55243858-48d4-4764-919b-43eefb040dda None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:44.173166 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55243858-48d4-4764-919b-43eefb040dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:44.195493 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-55243858-48d4-4764-919b-43eefb040dda None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:44.196786 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-55243858-48d4-4764-919b-43eefb040dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:44.201513 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:44.262233 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1423/2853] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:44.271506 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c5b58-a856-4c07-93df-92b2a507b413 None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:44.271886 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c5b58-a856-4c07-93df-92b2a507b413 None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:44.272213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c5b58-a856-4c07-93df-92b2a507b413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:44.293074 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-8f4c5b58-a856-4c07-93df-92b2a507b413 None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:44.294330 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8f4c5b58-a856-4c07-93df-92b2a507b413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:44.304027 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1431/2854] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:44.309287 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55243858-48d4-4764-919b-43eefb040dda None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:44.319800 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55243858-48d4-4764-919b-43eefb040dda None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:44.324934 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25480ded-4bd3-4637-90e9-78d10946cd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:44.329225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25480ded-4bd3-4637-90e9-78d10946cd33 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:44.329225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25480ded-4bd3-4637-90e9-78d10946cd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:44.329225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25480ded-4bd3-4637-90e9-78d10946cd33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:44.329225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25480ded-4bd3-4637-90e9-78d10946cd33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:43:44.329225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25480ded-4bd3-4637-90e9-78d10946cd33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:44.329225 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25480ded-4bd3-4637-90e9-78d10946cd33 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:44.343292 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1424/2855] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 15:43:44.347507 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c5b58-a856-4c07-93df-92b2a507b413 None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:44.357763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c5b58-a856-4c07-93df-92b2a507b413 None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:44.362441 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33cd813-f7a6-4230-898f-fe50cd449a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:44.363099 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cd813-f7a6-4230-898f-fe50cd449a2e admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:44.363382 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cd813-f7a6-4230-898f-fe50cd449a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:44.363704 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cd813-f7a6-4230-898f-fe50cd449a2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:44.364389 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33cd813-f7a6-4230-898f-fe50cd449a2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:44.366297 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33cd813-f7a6-4230-898f-fe50cd449a2e admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:44.521326 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1432/2856] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:43:44 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:43:44.526233 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25480ded-4bd3-4637-90e9-78d10946cd33 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:44.536317 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25480ded-4bd3-4637-90e9-78d10946cd33 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:44.605596 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f98ae-1fdb-477c-af77-0b85936b3515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:44.606306 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f98ae-1fdb-477c-af77-0b85936b3515 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:44.606593 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f98ae-1fdb-477c-af77-0b85936b3515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:44.606871 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f98ae-1fdb-477c-af77-0b85936b3515 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:44.607574 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f98ae-1fdb-477c-af77-0b85936b3515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:44.609496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f98ae-1fdb-477c-af77-0b85936b3515 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:44.751925 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1425/2857] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:43:44 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:43:44.755921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33cd813-f7a6-4230-898f-fe50cd449a2e admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:44.766398 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33cd813-f7a6-4230-898f-fe50cd449a2e admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:44.771035 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f9c62-70fe-4baf-8473-f51f7f131c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:44.771755 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f9c62-70fe-4baf-8473-f51f7f131c6f admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:44.771977 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f9c62-70fe-4baf-8473-f51f7f131c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:44.772250 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f9c62-70fe-4baf-8473-f51f7f131c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:44.772964 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f9c62-70fe-4baf-8473-f51f7f131c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:44.774685 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f9c62-70fe-4baf-8473-f51f7f131c6f admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:44.775964 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:44.785680 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1433/2858] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:43:44.795218 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f98ae-1fdb-477c-af77-0b85936b3515 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:44.806179 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f98ae-1fdb-477c-af77-0b85936b3515 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:44.810939 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65388ef9-6337-4669-a0f6-cbe11d52caf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:44.811603 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65388ef9-6337-4669-a0f6-cbe11d52caf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:44.811893 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65388ef9-6337-4669-a0f6-cbe11d52caf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:44.812175 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65388ef9-6337-4669-a0f6-cbe11d52caf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:44.812910 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65388ef9-6337-4669-a0f6-cbe11d52caf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:44.814676 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65388ef9-6337-4669-a0f6-cbe11d52caf6 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:44.825469 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1426/2859] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:43:44.829754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f9c62-70fe-4baf-8473-f51f7f131c6f admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:44.839710 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f9c62-70fe-4baf-8473-f51f7f131c6f admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:44.844215 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f39b1-c7f2-44f4-9221-85714f28fbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:44.845163 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f39b1-c7f2-44f4-9221-85714f28fbb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:44.845446 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f39b1-c7f2-44f4-9221-85714f28fbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:44.845720 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f39b1-c7f2-44f4-9221-85714f28fbb4 admin admin] PATH_INFO: `/v3/projects/0fcf74a861894146997f079b6e8c6776/users/ffe3f511c8504512a9bad3064539108e/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:44.862254 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f39b1-c7f2-44f4-9221-85714f28fbb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fcf74a861894146997f079b6e8c6776, user_id=ffe3f511c8504512a9bad3064539108e, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:44.864950 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f39b1-c7f2-44f4-9221-85714f28fbb4 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:44.967134 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1434/2860] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:43:44 2022] PUT /identity/v3/projects/0fcf74a861894146997f079b6e8c6776/users/ffe3f511c8504512a9bad3064539108e/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:44.978501 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23e812-3b87-4448-b845-3ec6d8f6deac None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:44.978795 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23e812-3b87-4448-b845-3ec6d8f6deac None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:44.979074 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23e812-3b87-4448-b845-3ec6d8f6deac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:45.012625 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-4a23e812-3b87-4448-b845-3ec6d8f6deac None None] MFA Rules not processed for user `ffe3f511c8504512a9bad3064539108e`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:45.013869 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-4a23e812-3b87-4448-b845-3ec6d8f6deac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:45.080563 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1427/2861] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:45.092057 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f39b1-c7f2-44f4-9221-85714f28fbb4 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:45.093870 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-0e6f39b1-c7f2-44f4-9221-85714f28fbb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:45.150167 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f39b1-c7f2-44f4-9221-85714f28fbb4 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:45.153893 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b2ab5-cc68-488f-b333-c5055e7825a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:45.154573 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b2ab5-cc68-488f-b333-c5055e7825a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:45.154858 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b2ab5-cc68-488f-b333-c5055e7825a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:45.155136 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b2ab5-cc68-488f-b333-c5055e7825a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:45.163394 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358b2ab5-cc68-488f-b333-c5055e7825a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:45.165473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358b2ab5-cc68-488f-b333-c5055e7825a7 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:45.174175 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1435/2862] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:43:48.269991 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a23e812-3b87-4448-b845-3ec6d8f6deac None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:48.276002 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:48.332572 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a23e812-3b87-4448-b845-3ec6d8f6deac None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:48.337598 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e260b-e432-429e-b68c-e439872982fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:48.338802 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e260b-e432-429e-b68c-e439872982fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:48.339088 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e260b-e432-429e-b68c-e439872982fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:48.339369 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e260b-e432-429e-b68c-e439872982fb admin admin] PATH_INFO: `/v3/users/ffe3f511c8504512a9bad3064539108e` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:48.340571 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e260b-e432-429e-b68c-e439872982fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffe3f511c8504512a9bad3064539108e)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:48.342496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e260b-e432-429e-b68c-e439872982fb admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:48.562284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ad9e260b-e432-429e-b68c-e439872982fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffe3f511c8504512a9bad3064539108e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:48.648151 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ad9e260b-e432-429e-b68c-e439872982fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffe3f511c8504512a9bad3064539108e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:48.648790 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ad9e260b-e432-429e-b68c-e439872982fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffe3f511c8504512a9bad3064539108e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:48.662381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ad9e260b-e432-429e-b68c-e439872982fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffe3f511c8504512a9bad3064539108e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:48.669352 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-ad9e260b-e432-429e-b68c-e439872982fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffe3f511c8504512a9bad3064539108e'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:48.675579 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1428/2863] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:43:48 2022] DELETE /identity/v3/users/ffe3f511c8504512a9bad3064539108e => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:48.680692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5aa58-e81c-460c-b2f5-37c8b48e3dac None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:48.681187 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5aa58-e81c-460c-b2f5-37c8b48e3dac None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:48.681292 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5aa58-e81c-460c-b2f5-37c8b48e3dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:48.699595 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-e6e5aa58-e81c-460c-b2f5-37c8b48e3dac None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:48.700812 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e6e5aa58-e81c-460c-b2f5-37c8b48e3dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:48.759031 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1436/2864] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:48.768962 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e260b-e432-429e-b68c-e439872982fb admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:48.770496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ad9e260b-e432-429e-b68c-e439872982fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:48.833157 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e260b-e432-429e-b68c-e439872982fb admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:48.836919 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33663c-cbed-452c-9e5e-35b6835de8f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:48.837657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33663c-cbed-452c-9e5e-35b6835de8f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:48.837946 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33663c-cbed-452c-9e5e-35b6835de8f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:48.838228 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33663c-cbed-452c-9e5e-35b6835de8f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:48.848071 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33663c-cbed-452c-9e5e-35b6835de8f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:48.849626 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33663c-cbed-452c-9e5e-35b6835de8f5 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:48.859545 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1429/2865] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:43:49.295666 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5aa58-e81c-460c-b2f5-37c8b48e3dac None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:49.297122 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e6e5aa58-e81c-460c-b2f5-37c8b48e3dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:49.310811 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5aa58-e81c-460c-b2f5-37c8b48e3dac None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:49.315484 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7f90e-6889-4d1b-b9b6-c23f0a541f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:49.316364 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7f90e-6889-4d1b-b9b6-c23f0a541f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:49.316720 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7f90e-6889-4d1b-b9b6-c23f0a541f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:49.316924 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7f90e-6889-4d1b-b9b6-c23f0a541f68 admin admin] PATH_INFO: `/v3/projects/0fcf74a861894146997f079b6e8c6776` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:49.318135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7f90e-6889-4d1b-b9b6-c23f0a541f68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fcf74a861894146997f079b6e8c6776)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:49.320170 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7f90e-6889-4d1b-b9b6-c23f0a541f68 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:49.448921 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c8e7f90e-6889-4d1b-b9b6-c23f0a541f68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fcf74a861894146997f079b6e8c6776', 'request_id': 'req-c8e7f90e-6889-4d1b-b9b6-c23f0a541f68'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:49.544872 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c8e7f90e-6889-4d1b-b9b6-c23f0a541f68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fcf74a861894146997f079b6e8c6776', 'request_id': 'req-c8e7f90e-6889-4d1b-b9b6-c23f0a541f68'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:49.552701 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c8e7f90e-6889-4d1b-b9b6-c23f0a541f68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fcf74a861894146997f079b6e8c6776', 'request_id': 'req-c8e7f90e-6889-4d1b-b9b6-c23f0a541f68'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:49.559262 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:49.564268 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c8e7f90e-6889-4d1b-b9b6-c23f0a541f68 admin admin] The token cache is being invalidate because project 0fcf74a861894146997f079b6e8c6776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:43:49.564568 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c8e7f90e-6889-4d1b-b9b6-c23f0a541f68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:49.565213 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-c8e7f90e-6889-4d1b-b9b6-c23f0a541f68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:43:49.566885 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1437/2866] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:43:49 2022] DELETE /identity/v3/projects/0fcf74a861894146997f079b6e8c6776 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:43:49.571792 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560eb7a-81fe-42dc-b5a0-1094a44fe1ad None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:49.572316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560eb7a-81fe-42dc-b5a0-1094a44fe1ad None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:49.572449 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560eb7a-81fe-42dc-b5a0-1094a44fe1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:49.591341 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-a560eb7a-81fe-42dc-b5a0-1094a44fe1ad None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:49.592545 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-a560eb7a-81fe-42dc-b5a0-1094a44fe1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:49.613990 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:43:49.652638 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1430/2867] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:49.660611 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd1aec-1cf0-407b-ba27-393f6b741eff None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:49.660894 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd1aec-1cf0-407b-ba27-393f6b741eff None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:49.661180 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd1aec-1cf0-407b-ba27-393f6b741eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:49.680160 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-eadd1aec-1cf0-407b-ba27-393f6b741eff None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:49.681384 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-eadd1aec-1cf0-407b-ba27-393f6b741eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:49.689377 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1438/2868] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:49.693748 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560eb7a-81fe-42dc-b5a0-1094a44fe1ad None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:49.704107 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560eb7a-81fe-42dc-b5a0-1094a44fe1ad None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:49.708811 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc820f06-8566-4981-8e59-d5334f4eedda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:49.709504 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc820f06-8566-4981-8e59-d5334f4eedda admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:49.721025 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc820f06-8566-4981-8e59-d5334f4eedda admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:49.721025 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc820f06-8566-4981-8e59-d5334f4eedda admin admin] PATH_INFO: `/v3/domains` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:49.721025 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc820f06-8566-4981-8e59-d5334f4eedda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 15:43:49.721025 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc820f06-8566-4981-8e59-d5334f4eedda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:49.721025 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc820f06-8566-4981-8e59-d5334f4eedda admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:49.725524 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1431/2869] 10.222.0.78 () {58 vars in 1152 bytes} [Wed Jul 20 15:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 15:43:49.730312 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd1aec-1cf0-407b-ba27-393f6b741eff None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:49.739684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd1aec-1cf0-407b-ba27-393f6b741eff None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:49.744358 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdc2de-e678-4416-b71a-0f09e886d93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:49.745013 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdc2de-e678-4416-b71a-0f09e886d93f admin admin] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:49.745328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdc2de-e678-4416-b71a-0f09e886d93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:49.745607 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdc2de-e678-4416-b71a-0f09e886d93f admin admin] PATH_INFO: `/v3/projects` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:49.746336 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbdc2de-e678-4416-b71a-0f09e886d93f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:49.748246 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbdc2de-e678-4416-b71a-0f09e886d93f admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:49.875292 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1439/2870] 10.222.0.78 () {60 vars in 1152 bytes} [Wed Jul 20 15:43:49 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:43:49.879760 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc820f06-8566-4981-8e59-d5334f4eedda admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:49.890326 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc820f06-8566-4981-8e59-d5334f4eedda admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:49.948788 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6209f8-8773-471a-8fa7-44e45dad4791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:49.949471 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6209f8-8773-471a-8fa7-44e45dad4791 admin admin] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:49.949757 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6209f8-8773-471a-8fa7-44e45dad4791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:49.950032 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6209f8-8773-471a-8fa7-44e45dad4791 admin admin] PATH_INFO: `/v3/users` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:49.950783 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6209f8-8773-471a-8fa7-44e45dad4791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:49.952733 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6209f8-8773-471a-8fa7-44e45dad4791 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:50.053968 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1432/2871] 10.222.0.78 () {60 vars in 1143 bytes} [Wed Jul 20 15:43:49 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 15:43:50.058215 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdc2de-e678-4416-b71a-0f09e886d93f admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:50.069269 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdc2de-e678-4416-b71a-0f09e886d93f admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:50.075457 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956b8dc-333a-444a-ba82-8bd719c301f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:50.076182 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956b8dc-333a-444a-ba82-8bd719c301f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:50.076469 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956b8dc-333a-444a-ba82-8bd719c301f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:50.076768 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956b8dc-333a-444a-ba82-8bd719c301f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:50.077507 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956b8dc-333a-444a-ba82-8bd719c301f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:50.079177 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956b8dc-333a-444a-ba82-8bd719c301f4 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:50.091671 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1440/2872] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:43:50.096181 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6209f8-8773-471a-8fa7-44e45dad4791 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:50.106805 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6209f8-8773-471a-8fa7-44e45dad4791 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:50.111550 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b456e-42c7-4164-bcf2-2b26bef99b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:50.112217 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b456e-42c7-4164-bcf2-2b26bef99b54 admin admin] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:50.112496 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b456e-42c7-4164-bcf2-2b26bef99b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:50.112774 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b456e-42c7-4164-bcf2-2b26bef99b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:50.113506 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b456e-42c7-4164-bcf2-2b26bef99b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:50.115169 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b456e-42c7-4164-bcf2-2b26bef99b54 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:50.126139 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1433/2873] 10.222.0.78 () {58 vars in 1121 bytes} [Wed Jul 20 15:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 15:43:50.130127 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956b8dc-333a-444a-ba82-8bd719c301f4 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:50.140600 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956b8dc-333a-444a-ba82-8bd719c301f4 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:50.145241 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab0b04-157c-44da-abc5-86cdc6e2d749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:50.147376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab0b04-157c-44da-abc5-86cdc6e2d749 admin admin] REQUEST_METHOD: `PUT` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:50.147376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab0b04-157c-44da-abc5-86cdc6e2d749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:50.147376 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab0b04-157c-44da-abc5-86cdc6e2d749 admin admin] PATH_INFO: `/v3/projects/14078d32fe294b2784fb0521b57db424/users/c72bddf22f6842a2bda756cf1a3a1573/roles/d85783606f6649f5b9465f407cafb610` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:50.165046 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab0b04-157c-44da-abc5-86cdc6e2d749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14078d32fe294b2784fb0521b57db424, user_id=c72bddf22f6842a2bda756cf1a3a1573, role_id=d85783606f6649f5b9465f407cafb610)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:50.167735 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab0b04-157c-44da-abc5-86cdc6e2d749 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:50.274249 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1441/2874] 10.222.0.78 () {60 vars in 1482 bytes} [Wed Jul 20 15:43:50 2022] PUT /identity/v3/projects/14078d32fe294b2784fb0521b57db424/users/c72bddf22f6842a2bda756cf1a3a1573/roles/d85783606f6649f5b9465f407cafb610 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:43:50.285834 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a6331-d1d2-4bdc-aeae-437cd622acc9 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:50.286195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a6331-d1d2-4bdc-aeae-437cd622acc9 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:50.286507 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a6331-d1d2-4bdc-aeae-437cd622acc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:50.314319 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-2b4a6331-d1d2-4bdc-aeae-437cd622acc9 None None] MFA Rules not processed for user `c72bddf22f6842a2bda756cf1a3a1573`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:50.315605 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-2b4a6331-d1d2-4bdc-aeae-437cd622acc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:50.382310 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1434/2875] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:43:50.393479 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab0b04-157c-44da-abc5-86cdc6e2d749 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:43:50.395453 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-42ab0b04-157c-44da-abc5-86cdc6e2d749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:50.453458 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab0b04-157c-44da-abc5-86cdc6e2d749 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:43:50.457207 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b34191-9978-4329-8850-37448d0f9148 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51e9bdab4164d87b7b0b11c505c2c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:43:50.457856 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b34191-9978-4329-8850-37448d0f9148 service cinder] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:50.458136 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b34191-9978-4329-8850-37448d0f9148 service cinder] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:50.458449 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b34191-9978-4329-8850-37448d0f9148 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:50.466991 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b34191-9978-4329-8850-37448d0f9148 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:43:50.468424 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b34191-9978-4329-8850-37448d0f9148 service cinder] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:43:50.477911 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1442/2876] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:43:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:43:52.313863 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239a679-a003-4804-9d3a-e66578d7d4fc None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:43:52.314399 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239a679-a003-4804-9d3a-e66578d7d4fc None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:43:52.314522 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239a679-a003-4804-9d3a-e66578d7d4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:43:52.334126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-5239a679-a003-4804-9d3a-e66578d7d4fc None None] MFA Rules not processed for user `c72bddf22f6842a2bda756cf1a3a1573`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:43:52.335399 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-5239a679-a003-4804-9d3a-e66578d7d4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:43:52.343202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1435/2877] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:43:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:44:00.474071 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:44:02.328890 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:44:02.688225 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b34191-9978-4329-8850-37448d0f9148 service cinder] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:44:02.689824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-e9b34191-9978-4329-8850-37448d0f9148 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:44:02.765975 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b34191-9978-4329-8850-37448d0f9148 service cinder] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:44:02.772094 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6203e11-ad25-4fc3-9e89-fef372a4cfb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:44:02.772771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6203e11-ad25-4fc3-9e89-fef372a4cfb2 service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:02.773060 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6203e11-ad25-4fc3-9e89-fef372a4cfb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:02.773344 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6203e11-ad25-4fc3-9e89-fef372a4cfb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:02.775340 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-a6203e11-ad25-4fc3-9e89-fef372a4cfb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:44:02.788697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6203e11-ad25-4fc3-9e89-fef372a4cfb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:44:02.790861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6203e11-ad25-4fc3-9e89-fef372a4cfb2 service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:44:02.803102 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1443/2878] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:44:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:44:08.775366 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c817c-e0a7-4a82-901f-5905f37d6ace None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:08.775852 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c817c-e0a7-4a82-901f-5905f37d6ace None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:08.775957 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c817c-e0a7-4a82-901f-5905f37d6ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:08.794643 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-105c817c-e0a7-4a82-901f-5905f37d6ace None None] MFA Rules not processed for user `4e75a82cc7e94702ba0080fc65d6eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:44:08.796638 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-105c817c-e0a7-4a82-901f-5905f37d6ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:44:08.852804 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1436/2879] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Jul 20 15:44:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:44:09.134709 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6203e11-ad25-4fc3-9e89-fef372a4cfb2 service nova] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:44:09.136254 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-a6203e11-ad25-4fc3-9e89-fef372a4cfb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:44:09.217214 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6203e11-ad25-4fc3-9e89-fef372a4cfb2 service nova] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:44:09.222139 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95cd9c-b6d3-4aee-862a-707ea6925e73 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0d719c0eb04a3b8a228c6ce70fdf98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:44:09.222816 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95cd9c-b6d3-4aee-862a-707ea6925e73 service swift] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:09.223092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95cd9c-b6d3-4aee-862a-707ea6925e73 service swift] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:09.223365 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95cd9c-b6d3-4aee-862a-707ea6925e73 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:09.232454 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95cd9c-b6d3-4aee-862a-707ea6925e73 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:44:09.234014 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95cd9c-b6d3-4aee-862a-707ea6925e73 service swift] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:44:09.242049 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1444/2880] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Jul 20 15:44:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:44:10.162855 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e0435-b39b-4bb5-b498-940881cc6586 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:10.163342 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e0435-b39b-4bb5-b498-940881cc6586 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:10.163444 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e0435-b39b-4bb5-b498-940881cc6586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:10.183518 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-386e0435-b39b-4bb5-b498-940881cc6586 None None] MFA Rules not processed for user `4e75a82cc7e94702ba0080fc65d6eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:44:10.184788 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-386e0435-b39b-4bb5-b498-940881cc6586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:44:10.192226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1437/2881] 10.222.0.78 () {58 vars in 967 bytes} [Wed Jul 20 15:44:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:44:10.204863 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95cd9c-b6d3-4aee-862a-707ea6925e73 service swift] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:44:10.215717 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95cd9c-b6d3-4aee-862a-707ea6925e73 service swift] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:44:10.221477 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719e09a-04b8-4314-9a70-477b29c9ced1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0d719c0eb04a3b8a228c6ce70fdf98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:44:10.221696 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719e09a-04b8-4314-9a70-477b29c9ced1 service swift] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:10.221982 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719e09a-04b8-4314-9a70-477b29c9ced1 service swift] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:10.222432 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719e09a-04b8-4314-9a70-477b29c9ced1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:10.232748 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0719e09a-04b8-4314-9a70-477b29c9ced1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:44:10.236719 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0719e09a-04b8-4314-9a70-477b29c9ced1 service swift] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:44:10.245254 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1445/2882] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Jul 20 15:44:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:44:11.179153 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36209b88-5eae-4676-a16e-bbfb570a890c None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:11.180195 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36209b88-5eae-4676-a16e-bbfb570a890c None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:11.180388 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36209b88-5eae-4676-a16e-bbfb570a890c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:11.202206 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-36209b88-5eae-4676-a16e-bbfb570a890c None None] MFA Rules not processed for user `4e75a82cc7e94702ba0080fc65d6eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:44:11.203738 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-36209b88-5eae-4676-a16e-bbfb570a890c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:44:11.211278 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1438/2883] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Jul 20 15:44:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 15:44:11.224297 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719e09a-04b8-4314-9a70-477b29c9ced1 service swift] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:44:11.235359 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719e09a-04b8-4314-9a70-477b29c9ced1 service swift] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:44:11.240401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ff72c-924b-46c2-8469-ff7c94f5bcfc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0d719c0eb04a3b8a228c6ce70fdf98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:44:11.241041 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ff72c-924b-46c2-8469-ff7c94f5bcfc service swift] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:11.241326 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ff72c-924b-46c2-8469-ff7c94f5bcfc service swift] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:11.241625 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ff72c-924b-46c2-8469-ff7c94f5bcfc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:11.251354 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ff72c-924b-46c2-8469-ff7c94f5bcfc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:44:11.252948 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ff72c-924b-46c2-8469-ff7c94f5bcfc service swift] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:44:11.261670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1446/2884] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Jul 20 15:44:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:44:12.035096 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec098475-36a1-4cfb-a5fe-2e8a44a7b6d9 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:12.035387 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec098475-36a1-4cfb-a5fe-2e8a44a7b6d9 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:12.035698 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec098475-36a1-4cfb-a5fe-2e8a44a7b6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:12.055058 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-ec098475-36a1-4cfb-a5fe-2e8a44a7b6d9 None None] MFA Rules not processed for user `4e75a82cc7e94702ba0080fc65d6eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:44:12.056187 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ec098475-36a1-4cfb-a5fe-2e8a44a7b6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:44:12.063585 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1439/2885] 10.222.0.78 () {58 vars in 967 bytes} [Wed Jul 20 15:44:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:44:12.076080 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ff72c-924b-46c2-8469-ff7c94f5bcfc service swift] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:44:12.087697 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ff72c-924b-46c2-8469-ff7c94f5bcfc service swift] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:44:12.092978 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd490903-1fd7-4077-9b9b-c2ce6d065fa6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0d719c0eb04a3b8a228c6ce70fdf98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:44:12.093678 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd490903-1fd7-4077-9b9b-c2ce6d065fa6 service swift] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:12.093966 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd490903-1fd7-4077-9b9b-c2ce6d065fa6 service swift] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:12.094402 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd490903-1fd7-4077-9b9b-c2ce6d065fa6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:12.104135 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd490903-1fd7-4077-9b9b-c2ce6d065fa6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:44:12.105763 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd490903-1fd7-4077-9b9b-c2ce6d065fa6 service swift] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:44:12.114627 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1447/2886] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Jul 20 15:44:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 15:44:15.419307 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec098475-36a1-4cfb-a5fe-2e8a44a7b6d9 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:44:15.483892 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec098475-36a1-4cfb-a5fe-2e8a44a7b6d9 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:44:15.488530 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c199cf0-afea-4e2d-8f94-0145bc6d0c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:44:15.489420 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c199cf0-afea-4e2d-8f94-0145bc6d0c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:15.489707 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c199cf0-afea-4e2d-8f94-0145bc6d0c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:15.490107 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c199cf0-afea-4e2d-8f94-0145bc6d0c0c admin admin] PATH_INFO: `/v3/users/c72bddf22f6842a2bda756cf1a3a1573` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:15.491381 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c199cf0-afea-4e2d-8f94-0145bc6d0c0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c72bddf22f6842a2bda756cf1a3a1573)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:44:15.493357 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c199cf0-afea-4e2d-8f94-0145bc6d0c0c admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:44:15.715955 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-7c199cf0-afea-4e2d-8f94-0145bc6d0c0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c72bddf22f6842a2bda756cf1a3a1573'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:44:15.824336 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-7c199cf0-afea-4e2d-8f94-0145bc6d0c0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c72bddf22f6842a2bda756cf1a3a1573'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:44:15.825096 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-7c199cf0-afea-4e2d-8f94-0145bc6d0c0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c72bddf22f6842a2bda756cf1a3a1573'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:44:15.838459 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-7c199cf0-afea-4e2d-8f94-0145bc6d0c0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c72bddf22f6842a2bda756cf1a3a1573'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:44:15.845346 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-7c199cf0-afea-4e2d-8f94-0145bc6d0c0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c72bddf22f6842a2bda756cf1a3a1573'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:44:15.851294 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1440/2887] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:44:15 2022] DELETE /identity/v3/users/c72bddf22f6842a2bda756cf1a3a1573 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:44:15.856031 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46776484-b053-495c-88ce-4c43e5ea16ab None None] REQUEST_METHOD: `POST` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:15.856513 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46776484-b053-495c-88ce-4c43e5ea16ab None None] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:15.856632 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46776484-b053-495c-88ce-4c43e5ea16ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:15.875128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.auth.core [None req-46776484-b053-495c-88ce-4c43e5ea16ab None None] MFA Rules not processed for user `6049ef02a0334b599d9582b3c2aa2ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:44:15.876609 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-46776484-b053-495c-88ce-4c43e5ea16ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:44:15.896513 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=100 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:44:15.936817 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1448/2888] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:44:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:44:15.947110 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c199cf0-afea-4e2d-8f94-0145bc6d0c0c admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:44:15.949477 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-7c199cf0-afea-4e2d-8f94-0145bc6d0c0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:44:15.955993 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:44:16.006881 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c199cf0-afea-4e2d-8f94-0145bc6d0c0c admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:44:16.011067 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98efec4-a9f2-4494-912d-d923d32f57b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:44:16.011721 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98efec4-a9f2-4494-912d-d923d32f57b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:16.012009 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98efec4-a9f2-4494-912d-d923d32f57b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:16.012290 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98efec4-a9f2-4494-912d-d923d32f57b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:16.021770 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98efec4-a9f2-4494-912d-d923d32f57b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:44:16.023226 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98efec4-a9f2-4494-912d-d923d32f57b6 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:44:16.033482 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1441/2889] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:44:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:44:16.311848 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46776484-b053-495c-88ce-4c43e5ea16ab None None] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:44:16.313374 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-46776484-b053-495c-88ce-4c43e5ea16ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:44:16.327401 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46776484-b053-495c-88ce-4c43e5ea16ab None None] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:44:16.332995 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900a383-2c8e-480d-b5b7-ec97a8a83d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:44:16.333853 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900a383-2c8e-480d-b5b7-ec97a8a83d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:16.334156 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900a383-2c8e-480d-b5b7-ec97a8a83d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:16.334462 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900a383-2c8e-480d-b5b7-ec97a8a83d53 admin admin] PATH_INFO: `/v3/projects/14078d32fe294b2784fb0521b57db424` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:16.335663 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900a383-2c8e-480d-b5b7-ec97a8a83d53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14078d32fe294b2784fb0521b57db424)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:44:16.337633 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900a383-2c8e-480d-b5b7-ec97a8a83d53 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:44:16.480420 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f900a383-2c8e-480d-b5b7-ec97a8a83d53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14078d32fe294b2784fb0521b57db424', 'request_id': 'req-f900a383-2c8e-480d-b5b7-ec97a8a83d53'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:44:16.687253 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f900a383-2c8e-480d-b5b7-ec97a8a83d53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14078d32fe294b2784fb0521b57db424', 'request_id': 'req-f900a383-2c8e-480d-b5b7-ec97a8a83d53'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:44:16.696428 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f900a383-2c8e-480d-b5b7-ec97a8a83d53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14078d32fe294b2784fb0521b57db424', 'request_id': 'req-f900a383-2c8e-480d-b5b7-ec97a8a83d53'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:44:16.709431 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f900a383-2c8e-480d-b5b7-ec97a8a83d53 admin admin] The token cache is being invalidate because project 14078d32fe294b2784fb0521b57db424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:44:16.720721 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f900a383-2c8e-480d-b5b7-ec97a8a83d53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:44:16.720721 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-f900a383-2c8e-480d-b5b7-ec97a8a83d53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:44:16.720721 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1449/2890] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:44:16 2022] DELETE /identity/v3/projects/14078d32fe294b2784fb0521b57db424 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:44:26.109488 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=24 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:44:26.705205 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=43,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:44:37.514951 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239918d-9b4e-4d5d-9138-e0b4dd4d9030 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:44:37.515486 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239918d-9b4e-4d5d-9138-e0b4dd4d9030 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:44:37.515486 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239918d-9b4e-4d5d-9138-e0b4dd4d9030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:44:37.533229 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-d239918d-9b4e-4d5d-9138-e0b4dd4d9030 None None] MFA Rules not processed for user `cb4dc884b7eb4aa8a0adcdb46c5f9586`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:44:37.534488 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d239918d-9b4e-4d5d-9138-e0b4dd4d9030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:44:37.588753 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1442/2891] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:44:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:44:47.580587 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=25 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:45:50.864912 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900a383-2c8e-480d-b5b7-ec97a8a83d53 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:45:50.866529 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-f900a383-2c8e-480d-b5b7-ec97a8a83d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:50.942743 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900a383-2c8e-480d-b5b7-ec97a8a83d53 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:45:50.948131 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5473e3-b004-46ee-a38f-ec436f9c95e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2192547babde4449a46f9cb37ed64524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:45:50.948791 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5473e3-b004-46ee-a38f-ec436f9c95e6 service placement] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:45:50.949075 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5473e3-b004-46ee-a38f-ec436f9c95e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:45:50.949378 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5473e3-b004-46ee-a38f-ec436f9c95e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:45:50.951223 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ed5473e3-b004-46ee-a38f-ec436f9c95e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:45:50.963490 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5473e3-b004-46ee-a38f-ec436f9c95e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:45:50.965108 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5473e3-b004-46ee-a38f-ec436f9c95e6 service placement] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:45:50.975416 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1450/2892] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:45:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:46:00.970984 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=32 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:35.148338 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314aa43-c1ad-4771-992c-6cd5ef078e35 None None] REQUEST_METHOD: `POST` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:35.149042 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314aa43-c1ad-4771-992c-6cd5ef078e35 None None] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:35.149316 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314aa43-c1ad-4771-992c-6cd5ef078e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:35.175068 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.auth.core [None req-f314aa43-c1ad-4771-992c-6cd5ef078e35 None None] MFA Rules not processed for user `cb4dc884b7eb4aa8a0adcdb46c5f9586`. Rule list: `[]` (Enabled: `True`). {{(pid=80509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 15:46:35.177324 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f314aa43-c1ad-4771-992c-6cd5ef078e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:35.190070 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1443/2893] 10.222.0.78 () {58 vars in 957 bytes} [Wed Jul 20 15:46:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 15:46:38.699820 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5473e3-b004-46ee-a38f-ec436f9c95e6 service placement] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:38.702056 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-ed5473e3-b004-46ee-a38f-ec436f9c95e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:38.806881 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5473e3-b004-46ee-a38f-ec436f9c95e6 service placement] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:38.813100 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a29309-48ce-4026-af2f-56a2000cc5b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:38.818815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a29309-48ce-4026-af2f-56a2000cc5b4 service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:38.818815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a29309-48ce-4026-af2f-56a2000cc5b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:38.818815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a29309-48ce-4026-af2f-56a2000cc5b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:38.818815 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-d9a29309-48ce-4026-af2f-56a2000cc5b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:38.827172 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a29309-48ce-4026-af2f-56a2000cc5b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:38.828747 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a29309-48ce-4026-af2f-56a2000cc5b4 service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:38.838155 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1451/2894] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:46:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:46:45.166667 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=3 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:48.833684 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=34 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:46:50.484064 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314aa43-c1ad-4771-992c-6cd5ef078e35 None None] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:50.485527 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-f314aa43-c1ad-4771-992c-6cd5ef078e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:50.552164 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314aa43-c1ad-4771-992c-6cd5ef078e35 None None] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:50.559476 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0656b8-d901-4f11-8552-ff03cb9350e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:50.560545 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0656b8-d901-4f11-8552-ff03cb9350e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:50.561516 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0656b8-d901-4f11-8552-ff03cb9350e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:50.561930 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0656b8-d901-4f11-8552-ff03cb9350e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:50.570407 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ac0656b8-d901-4f11-8552-ff03cb9350e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:50.588548 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0656b8-d901-4f11-8552-ff03cb9350e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:50.590513 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0656b8-d901-4f11-8552-ff03cb9350e5 service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:50.604500 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1444/2895] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:46:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:46:51.827771 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a29309-48ce-4026-af2f-56a2000cc5b4 service nova] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:46:51.838380 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a29309-48ce-4026-af2f-56a2000cc5b4 service nova] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:46:51.842476 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd7686-9c5a-4169-a849-fd977f59a989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:46:51.846172 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd7686-9c5a-4169-a849-fd977f59a989 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:46:51.846172 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd7686-9c5a-4169-a849-fd977f59a989 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:46:51.846172 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd7686-9c5a-4169-a849-fd977f59a989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:46:51.846172 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-defd7686-9c5a-4169-a849-fd977f59a989 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:46:51.860350 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd7686-9c5a-4169-a849-fd977f59a989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:46:51.861829 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd7686-9c5a-4169-a849-fd977f59a989 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:46:51.873983 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1452/2896] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:46:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:47:00.598177 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=26 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:47:01.868142 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=6 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:47:53.294610 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0656b8-d901-4f11-8552-ff03cb9350e5 service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:47:53.296346 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-ac0656b8-d901-4f11-8552-ff03cb9350e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:53.308091 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0656b8-d901-4f11-8552-ff03cb9350e5 service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:47:53.312003 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e1388-01ad-413d-90c0-0dbbb3b54b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:47:53.312657 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e1388-01ad-413d-90c0-0dbbb3b54b1c service neutron] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:47:53.312938 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e1388-01ad-413d-90c0-0dbbb3b54b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:47:53.313256 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e1388-01ad-413d-90c0-0dbbb3b54b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:47:53.315257 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-333e1388-01ad-413d-90c0-0dbbb3b54b1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:47:53.326082 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e1388-01ad-413d-90c0-0dbbb3b54b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:47:53.327536 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e1388-01ad-413d-90c0-0dbbb3b54b1c service neutron] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:47:53.336866 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1445/2897] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:47:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:48:03.332183 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=6 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:32.270034 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd7686-9c5a-4169-a849-fd977f59a989 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:32.283974 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd7686-9c5a-4169-a849-fd977f59a989 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:32.291514 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac6d95-70a1-4a97-8c8a-50a9825896a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:32.292700 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac6d95-70a1-4a97-8c8a-50a9825896a7 service nova] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:32.294692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac6d95-70a1-4a97-8c8a-50a9825896a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:32.294692 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac6d95-70a1-4a97-8c8a-50a9825896a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:32.296896 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-a5ac6d95-70a1-4a97-8c8a-50a9825896a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:32.319906 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac6d95-70a1-4a97-8c8a-50a9825896a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:32.323883 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac6d95-70a1-4a97-8c8a-50a9825896a7 service nova] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:32.343261 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1453/2898] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:48:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:48:35.517321 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e1388-01ad-413d-90c0-0dbbb3b54b1c service neutron] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:35.531852 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e1388-01ad-413d-90c0-0dbbb3b54b1c service neutron] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:35.537592 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf0fcf-2946-459e-9c7b-6ced4c2c50e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4a24d5a375945779fa626fae25ac237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:35.538301 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf0fcf-2946-459e-9c7b-6ced4c2c50e9 service nova] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:35.538617 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf0fcf-2946-459e-9c7b-6ced4c2c50e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:35.538870 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf0fcf-2946-459e-9c7b-6ced4c2c50e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:35.540863 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e7bf0fcf-2946-459e-9c7b-6ced4c2c50e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:35.555481 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf0fcf-2946-459e-9c7b-6ced4c2c50e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:35.557126 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf0fcf-2946-459e-9c7b-6ced4c2c50e9 service nova] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:35.568777 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1446/2899] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:48:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:48:42.334137 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=6 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:45.563731 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=6 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:48.232828 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac6d95-70a1-4a97-8c8a-50a9825896a7 service nova] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:48.246481 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac6d95-70a1-4a97-8c8a-50a9825896a7 service nova] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:48.252487 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf329ca-1a3c-4f1b-96b2-e27dc909f632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b57530f18d9c4d328f8014db736d8de9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:48.253520 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf329ca-1a3c-4f1b-96b2-e27dc909f632 service neutron] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:48.253975 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf329ca-1a3c-4f1b-96b2-e27dc909f632 service neutron] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:48.254946 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf329ca-1a3c-4f1b-96b2-e27dc909f632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:48.259326 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-8bf329ca-1a3c-4f1b-96b2-e27dc909f632 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:48.343473 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf329ca-1a3c-4f1b-96b2-e27dc909f632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:48.348824 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf329ca-1a3c-4f1b-96b2-e27dc909f632 service neutron] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:48.363535 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1454/2900] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:48:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 15:48:52.223073 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf0fcf-2946-459e-9c7b-6ced4c2c50e9 service nova] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:52.224893 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-e7bf0fcf-2946-459e-9c7b-6ced4c2c50e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:52.240396 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf0fcf-2946-459e-9c7b-6ced4c2c50e9 service nova] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:52.246463 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb21ef-606e-47ca-9bcc-8af9f09e6fcb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2192547babde4449a46f9cb37ed64524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:52.247141 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb21ef-606e-47ca-9bcc-8af9f09e6fcb service placement] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:52.247483 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb21ef-606e-47ca-9bcc-8af9f09e6fcb service placement] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:52.247927 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb21ef-606e-47ca-9bcc-8af9f09e6fcb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:52.250053 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-46eb21ef-606e-47ca-9bcc-8af9f09e6fcb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:52.263875 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb21ef-606e-47ca-9bcc-8af9f09e6fcb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:52.265541 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb21ef-606e-47ca-9bcc-8af9f09e6fcb service placement] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:52.276593 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1447/2901] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:48:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:48:56.976081 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf329ca-1a3c-4f1b-96b2-e27dc909f632 service neutron] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:56.995643 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf329ca-1a3c-4f1b-96b2-e27dc909f632 service neutron] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:57.001092 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194973ac-a458-4c39-ab04-f7932df0a13a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2192547babde4449a46f9cb37ed64524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:57.001754 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194973ac-a458-4c39-ab04-f7932df0a13a service placement] REQUEST_METHOD: `GET` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:57.002245 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194973ac-a458-4c39-ab04-f7932df0a13a service placement] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:57.002674 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194973ac-a458-4c39-ab04-f7932df0a13a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:57.005021 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-194973ac-a458-4c39-ab04-f7932df0a13a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:57.019047 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194973ac-a458-4c39-ab04-f7932df0a13a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:57.020863 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194973ac-a458-4c39-ab04-f7932df0a13a service placement] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:57.033200 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1455/2902] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:48:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:48:57.150344 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb21ef-606e-47ca-9bcc-8af9f09e6fcb service placement] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:57.153284 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-46eb21ef-606e-47ca-9bcc-8af9f09e6fcb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:57.175900 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb21ef-606e-47ca-9bcc-8af9f09e6fcb service placement] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:57.183736 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cbea1-6cb9-4d94-808b-5ff6dab3384f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:57.185218 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cbea1-6cb9-4d94-808b-5ff6dab3384f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:57.185727 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cbea1-6cb9-4d94-808b-5ff6dab3384f admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:57.186227 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cbea1-6cb9-4d94-808b-5ff6dab3384f admin admin] PATH_INFO: `/v3/users/cb4dc884b7eb4aa8a0adcdb46c5f9586` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:57.188355 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cbea1-6cb9-4d94-808b-5ff6dab3384f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb4dc884b7eb4aa8a0adcdb46c5f9586)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:57.192433 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cbea1-6cb9-4d94-808b-5ff6dab3384f admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:57.421906 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-804cbea1-6cb9-4d94-808b-5ff6dab3384f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb4dc884b7eb4aa8a0adcdb46c5f9586'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.497167 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-804cbea1-6cb9-4d94-808b-5ff6dab3384f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb4dc884b7eb4aa8a0adcdb46c5f9586'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.498196 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-804cbea1-6cb9-4d94-808b-5ff6dab3384f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb4dc884b7eb4aa8a0adcdb46c5f9586'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.514512 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-804cbea1-6cb9-4d94-808b-5ff6dab3384f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb4dc884b7eb4aa8a0adcdb46c5f9586'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.526432 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-804cbea1-6cb9-4d94-808b-5ff6dab3384f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb4dc884b7eb4aa8a0adcdb46c5f9586'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:57.536393 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1448/2903] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:48:57 2022] DELETE /identity/v3/users/cb4dc884b7eb4aa8a0adcdb46c5f9586 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:48:57.826410 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194973ac-a458-4c39-ab04-f7932df0a13a service placement] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:57.841863 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194973ac-a458-4c39-ab04-f7932df0a13a service placement] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:57.912083 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7211f79-4dde-43c1-827a-8c73dbecc5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:57.913128 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7211f79-4dde-43c1-827a-8c73dbecc5d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:57.913526 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7211f79-4dde-43c1-827a-8c73dbecc5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:57.913918 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7211f79-4dde-43c1-827a-8c73dbecc5d6 admin admin] PATH_INFO: `/v3/projects/68372ee91fb843108d3b0cb1c99cd186` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:57.915784 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7211f79-4dde-43c1-827a-8c73dbecc5d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68372ee91fb843108d3b0cb1c99cd186)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:57.917720 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7211f79-4dde-43c1-827a-8c73dbecc5d6 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:58.049116 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-a7211f79-4dde-43c1-827a-8c73dbecc5d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68372ee91fb843108d3b0cb1c99cd186', 'request_id': 'req-a7211f79-4dde-43c1-827a-8c73dbecc5d6'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:58.153861 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-a7211f79-4dde-43c1-827a-8c73dbecc5d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68372ee91fb843108d3b0cb1c99cd186', 'request_id': 'req-a7211f79-4dde-43c1-827a-8c73dbecc5d6'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:58.160923 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-a7211f79-4dde-43c1-827a-8c73dbecc5d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68372ee91fb843108d3b0cb1c99cd186', 'request_id': 'req-a7211f79-4dde-43c1-827a-8c73dbecc5d6'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:58.172112 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-a7211f79-4dde-43c1-827a-8c73dbecc5d6 admin admin] The token cache is being invalidate because project 68372ee91fb843108d3b0cb1c99cd186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:48:58.172398 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-a7211f79-4dde-43c1-827a-8c73dbecc5d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:58.173058 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-a7211f79-4dde-43c1-827a-8c73dbecc5d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:58.174806 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1456/2904] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:48:57 2022] DELETE /identity/v3/projects/68372ee91fb843108d3b0cb1c99cd186 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:48:58.179865 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cbea1-6cb9-4d94-808b-5ff6dab3384f admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:58.181469 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-804cbea1-6cb9-4d94-808b-5ff6dab3384f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:58.195973 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cbea1-6cb9-4d94-808b-5ff6dab3384f admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:58.270960 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5352d04-824b-484c-988d-53ba515d4144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:58.272380 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5352d04-824b-484c-988d-53ba515d4144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:58.272853 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5352d04-824b-484c-988d-53ba515d4144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:58.273297 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5352d04-824b-484c-988d-53ba515d4144 admin admin] PATH_INFO: `/v3/users/8cddba4b99e542499f27e0bf14f012b3` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:58.275762 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5352d04-824b-484c-988d-53ba515d4144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cddba4b99e542499f27e0bf14f012b3)` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:58.278923 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5352d04-824b-484c-988d-53ba515d4144 admin admin] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:58.511457 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d5352d04-824b-484c-988d-53ba515d4144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cddba4b99e542499f27e0bf14f012b3'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:58.610897 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d5352d04-824b-484c-988d-53ba515d4144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cddba4b99e542499f27e0bf14f012b3'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:58.611959 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d5352d04-824b-484c-988d-53ba515d4144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cddba4b99e542499f27e0bf14f012b3'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:58.618863 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:58.627095 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d5352d04-824b-484c-988d-53ba515d4144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cddba4b99e542499f27e0bf14f012b3'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:58.634629 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.notifications [None req-d5352d04-824b-484c-988d-53ba515d4144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cddba4b99e542499f27e0bf14f012b3'} {{(pid=80509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:58.641515 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1449/2905] 10.222.0.78 () {58 vars in 1223 bytes} [Wed Jul 20 15:48:58 2022] DELETE /identity/v3/users/8cddba4b99e542499f27e0bf14f012b3 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 15:48:59.021106 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7211f79-4dde-43c1-827a-8c73dbecc5d6 admin admin] Authenticating user token {{(pid=80508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:48:59.023328 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.fernet_utils [None req-a7211f79-4dde-43c1-827a-8c73dbecc5d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:48:59.050658 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7211f79-4dde-43c1-827a-8c73dbecc5d6 admin admin] Validating token access rules against request {{(pid=80508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:48:59.065161 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:48:59.148609 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8943a3f-d610-4e13-afab-68b5440f7453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6049ef02a0334b599d9582b3c2aa2ebf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'effa579779b9494f8b1dd5831edaae25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:48:59.150109 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8943a3f-d610-4e13-afab-68b5440f7453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:48:59.151142 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8943a3f-d610-4e13-afab-68b5440f7453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:48:59.151780 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8943a3f-d610-4e13-afab-68b5440f7453 admin admin] PATH_INFO: `/v3/projects/980fc587eb054036b5d7cde3f4d2a625` {{(pid=80508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:48:59.153948 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8943a3f-d610-4e13-afab-68b5440f7453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=980fc587eb054036b5d7cde3f4d2a625)` {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:48:59.157518 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8943a3f-d610-4e13-afab-68b5440f7453 admin admin] RBAC: Authorization granted {{(pid=80508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:48:59.331108 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b8943a3f-d610-4e13-afab-68b5440f7453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '980fc587eb054036b5d7cde3f4d2a625', 'request_id': 'req-b8943a3f-d610-4e13-afab-68b5440f7453'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:59.421972 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b8943a3f-d610-4e13-afab-68b5440f7453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '980fc587eb054036b5d7cde3f4d2a625', 'request_id': 'req-b8943a3f-d610-4e13-afab-68b5440f7453'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:59.432158 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b8943a3f-d610-4e13-afab-68b5440f7453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '980fc587eb054036b5d7cde3f4d2a625', 'request_id': 'req-b8943a3f-d610-4e13-afab-68b5440f7453'} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:59.444383 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b8943a3f-d610-4e13-afab-68b5440f7453 admin admin] The token cache is being invalidate because project 980fc587eb054036b5d7cde3f4d2a625 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 15:48:59.444670 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b8943a3f-d610-4e13-afab-68b5440f7453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:59.445321 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG keystone.notifications [None req-b8943a3f-d610-4e13-afab-68b5440f7453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 15:48:59.446938 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: [pid: 80508|app: 0|req: 1457/2906] 10.222.0.78 () {58 vars in 1232 bytes} [Wed Jul 20 15:48:59 2022] DELETE /identity/v3/projects/980fc587eb054036b5d7cde3f4d2a625 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 15:49:08.973752 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=9,keystone:DELETE=1 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:49:09.441078 dv-cinder-819149-12-iscsi devstack@keystone.service[80508]: DEBUG dbcounter [-] [80508] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:50:55.815390 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5352d04-824b-484c-988d-53ba515d4144 admin admin] Authenticating user token {{(pid=80509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 15:50:55.816911 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-d5352d04-824b-484c-988d-53ba515d4144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:50:55.976035 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5352d04-824b-484c-988d-53ba515d4144 admin admin] Validating token access rules against request {{(pid=80509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 15:50:56.039394 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833a461-6b53-427d-bc20-af35c267ae04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2192547babde4449a46f9cb37ed64524', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdae1cb71dd449b8a18062b7b52889e4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 15:50:56.039913 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833a461-6b53-427d-bc20-af35c267ae04 service placement] REQUEST_METHOD: `GET` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 15:50:56.040202 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833a461-6b53-427d-bc20-af35c267ae04 service placement] SCRIPT_NAME: `/identity` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 15:50:56.040482 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833a461-6b53-427d-bc20-af35c267ae04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 15:50:56.042491 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.fernet_utils [None req-1833a461-6b53-427d-bc20-af35c267ae04 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 15:50:56.055063 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833a461-6b53-427d-bc20-af35c267ae04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 15:50:56.056641 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833a461-6b53-427d-bc20-af35c267ae04 service placement] RBAC: Authorization granted {{(pid=80509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 15:50:56.067582 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: [pid: 80509|app: 0|req: 1450/2907] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Jul 20 15:50:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 15:51:06.063232 dv-cinder-819149-12-iscsi devstack@keystone.service[80509]: DEBUG dbcounter [-] [80509] Writing DB stats keystone:SELECT=89 {{(pid=80509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}